Cathy O'Neil (Catherine "Cathy" Helen O'Neil) Quotes

I know how models are built, because I build them myself, so I know that I'm embedding my values into every single algorithm I create and I am projecting my agenda onto those algorithms.

Cathy O'Neil

• Values
• Models
• Agendas
• Algorithms
• Embedding

• Previous
• Next

Quotes to Explore

• We've learned our lesson with finance because they made a huge goddamn explosion that almost shut down the world. But the thing I realized is that there might never be an explosion on the scale of the financial crisis happening with big data.

  

  Cathy O'Neil
  • World
  • Financial
  • Crisis

• I think there's inherently an issue that models will literally never be able to handle, which is that when somebody comes along with a new way of doing something that's really excellent, the models will not recognize it. They only know how to recognize excellence when they can measure it somehow.

  

  Cathy O'Neil
  • Able
  • Issues
  • Excellence

• People are starting to be very skeptical of the Facebook algorithm and all kinds of data surveillance.

  

  Cathy O'Neil
  • Kind
  • Starting
  • Data

• I don't think anybody's ever notified that they were sentenced to an extra two years because their recidivism score had been high, or notified that this beat cop happened to be in their neighborhood checking people's pockets for pot because of a predictive policing algorithm. That's just not how it works.

  

  Cathy O'Neil
  • People
  • Beats
  • Neighborhood

• The NSA buys data from private companies, so the private companies are the source of all this stuff.

  

  Cathy O'Neil
  • Company
  • Stuff
  • Source

• The Facebook algorithm designers chose to let us see what our friends are talking about. They chose to show us, in some sense, more of the same. And that is the design decision that they could have decided differently. They could have said, "We're going to show you stuff that you've probably never seen before." I think they probably optimized their algorithm to make the most amount of money, and that probably meant showing people stuff that they already sort of agreed with, or were more likely to agree with.

  

  Cathy O'Neil
  • Decision
  • Design
  • Agree

• The national conversation around white entitlement, around institutionalized racism, the Black Lives Matter movement, I think, came about in large part because of the widening and broadening of our understanding of inequality. That conversation was begun by Occupy.

  

  Cathy O'Neil
  • Black
  • Understanding
  • Racism

• Obviously the more transparency we have as auditors, the more we can get, but the main goal is to understand important characteristics about a black box algorithm without necessarily having to understand every single granular detail of the algorithm.

  

  Cathy O'Neil
  • Important
  • Black
  • Goal

• I would argue that one of the major problems with our blind trust in algorithms is that we can propagate discriminatory patterns without acknowledging any kind of intent.

  

  Cathy O'Neil
  • Kind
  • Problem
  • Blind

• I set up a company, an algorithmic auditing company myself. I have no clients.

  

  Cathy O'Neil
  • Company
  • Clients

• We can't just throw something out there and assume it works just because it has math in it.

  

  Cathy O'Neil
  • Math
  • Assuming

• It's a standard thing you hear from startup people - that their product is somehow improving the world. And if you follow the reasoning, you will get somewhere, and I'll tell you where you get: You'll get to the description of what happens to the winners under the system that they're building.

  

  Cathy O'Neil
  • World
  • Building
  • Standards

• By construction, the world of big data is siloed and segmented and segregated so that successful people, like myself - technologists, well-educated white people, for the most part - benefit from big data, and it's the people on the other side of the economic spectrum, especially people of color, who suffer from it. They suffer from it individually, at different times, at different moments. They never get a clear explanation of what actually happened to them because all these scores are secret and sometimes they don't even know they're being scored.

  

  Cathy O'Neil
  • People
  • World
  • Different

• Micro-targeting is the ability for a campaign to profile you, to know much more about you than you know about it, and then to choose exactly what to show you.

  

  Cathy O'Neil
  • Knows
  • Ability
  • Shows

• Because of my experience in Occupy, instead of asking the question, "Who will benefit from this system I'm implementing with the data?" I started to ask the question, "What will happen to the most vulnerable?" Or "Who is going to lose under this system? How will this affect the worst-off person?" Which is a very different question from "How does this improve certain people's lives?"

  

  Cathy O'Neil
  • Different
  • Certain
  • Worst

• So much of our society as a whole is gearing us to maximize our salary or bonus. Basically, we just think in terms of money. Or, if not money, then, if you're in academia, it's prestige. It's a different kind of currency. And there's this unmeasured dimension of all jobs, which is whether it's improving the world.

  

  Cathy O'Neil
  • World
  • Different
  • Kind

• With recidivism algorithms, for example, I worry about racist outcomes. With personality tests for hiring, I worry about filtering out people with mental health problems from jobs. And with a teacher value-added model algorithm used in New York City to score teachers, I worry literally that it's not meaningful. That it's almost a random number generator.

  

  Cathy O'Neil
  • People
  • Problem
  • Worry

• There's less of a connection for a lot of people between the technical decisions we make and the ethical ramifications we are responsible for.

  

  Cathy O'Neil
  • Decision
  • Responsible
  • Connections

• An insurance company might say, "Tell us more about yourself so your premiums can go down." When they say that, they're addressing the winners, not the losers.

  

  Cathy O'Neil
  • Company
  • Might
  • Winner

• There might never be that moment when everyone says, "Oh my God, big data is awful."

  

  Cathy O'Neil
  • Moments
  • Awful

• People felt like they were friends with Google, and they believed in the "Do No Evil" thing that Google said. They trusted Google more than they trusted the government, and I never understood that.

  

  Cathy O'Neil
  • People
  • Government
  • Evil

• The disconnect I was experiencing was that people hated Wall Street, but they loved tech.

  

  Cathy O'Neil
  • People
  • Wall
  • Streets

• We don't let a car company just throw out a car and start driving it around without checking that the wheels are fastened on. We know that would result in death; but for some reason we have no hesitation at throwing out some algorithms untested and unmonitored even when they're making very important life-and-death decisions.

  

  Cathy O'Neil
  • Important
  • Reason
  • Decision

• I know how models are built, because I build them myself, so I know that I'm embedding my values into every single algorithm I create and I am projecting my agenda onto those algorithms.

  

  Cathy O'Neil
  • Values
  • Models
  • Agendas