DataKind's Founder & Executive Director, Jake Porway, and Cathy O'Neil recently caught-up in New York City for a chat. Here's some of what they spoke about..
Cathy: I was inspired to write Weapons of Math Destructionbecause I was seeing an insidious pattern of careless and destructive algorithmic design that I thought would go unnoticed until it had done extensive damage. What if we didn't get the word out to people and let them know we need to change tack?
Cathy: Both! AI has been able to do a lot in narrow ways like winning at chess and Go. In bigger, more complicated social settings, AI has quite possibly done more harm than good, at least when the consequences of getting things wrong are real human suffering.
Cathy: I liked the idea of helping people look up their food stamp card balances. That's not really AI but at least it's aimed at helping the vulnerable rather than exploiting them.
Cathy: The idea that someday we might have our anti-discrimination laws enforced in algorithmic contexts. So far we don't. Secondarily, I'm always happy when algorithmic harm comes to public light, because it's a moment when we can start learning skepticism and make demands for scientific evidence that a given automated system is legal and does not cause negative externalities.