Ye Olde Natural Philosophy Discussion Group
Reviews and comments on
Cathy O’Neil:
Weapons of Math Destruction: How Big Data
Increases Inequality and Threatens Democracy [2017]
Our group had generally positive views of this book, which warns the public about how corporations, colleges and government agencies are now using computer algorithms to target us with ads; to determine whether we get into college; to decide whether or not we get hired, promoted or fired; to determine whether or not we can get insurance (and how much we will have to pay for it); to determine whether we are likely to get arrested and the length of our prison terms if we get convicted of a crime; and in so many other ways. It argues that these algorithms have built-in biases and unfairness, even if this is usually not actually intended by those who create and use them. On a scale of 0 to 10 we gave the book a rating of 6.67—which is not bad, considering that we tend to be tough graders.
Kirby commented that the book was not what he expected. “I really wanted to like it, but didn’t,” he said (though he still rated the book a 6). “I understand her issues and she made her point.” But he felt that in some cases the algorithms O’Neil talked about were really not all that unfair. With car insurance, for example, if you live in a bad neighborhood you really are more likely to have your car damaged or broken into, and therefore, says Kirby, the insurance company really does have the right (and responsibility?) to charge more for car insurance in your case.
[Editorial interjection: Actually, O’Neil gave some good general arguments against the sort of objection that Kirby raised above. For example:
“As insurance companies learn more about us, they’ll be able to pinpoint those who appear to be the riskiest customers and then either drive their rates to the stratosphere or, where legal, deny them coverage. This is a far cry from insurance’s original purpose, which is to help society balance its risk. In a targeted world, we no longer pay the average. Instead, we’re saddled with anticipated costs. Instead of smoothing out life’s bumps, insurance companies will demand payment for those bumps in advance. This undermines the point of insurance, and the hits will fall especially hard on those who can least afford them.” (p. 171)
[In an ideal world (for insurance companies!) they would know in advance exactly how much they will ever pay out to every individual policy holder, and then charge that person in advance that precise amount plus their pro-rated profit. But in that case it would be in no one’s interests to actually buy insurance. (Why not just wait for the damage to happen, pay for it then, and avoid paying the extra amount for the company’s profit?!) Such an “ideal world” will never completely exist, but the more that data mining, spying on individuals, and algorithms allow insurance companies to approximate it, the less sense it makes for people to buy insurance at all. The deeper the knowledge by the insurance company of the actual risk, the more likely it is that you will only end up paying out more by buying insurance. As O’Neil says, the entire point of having insurance is to spread the risk of possibly catastrophic large losses equally to everyone. The “socialization of risk” is the entire point of it all. (This is why the most rational form of insurance is through a government agency which automatically covers everybody, doesn’t need to waste money on advertising and big managerial salaries, doesn’t need to generate profits for stock holders, etc.) Moreover, if there is any deviation from the average premium for individual policy holders it should only be because of genuine faults of their own responsibility, not because of where they are essentially forced to live, etc. This is why Kirby’s point of view here represents a bourgeois individualist perspective, and indeed promotes some serious inequality and unfairness. —S.H.]
[Returning the floor to Kirby:] “She over-generalized,” said Kirby, though “she saved herself in her last few chapters. But she just made statements that are false. But on the plus side, it was an easy book to read. I don’t believe all her examples, but she made some interesting points.”
Rosie said that “although I agree with everything Kirby just said” she still thought it was a good book focused on “important issues”. She rated it an 8.
Rich said he liked the book. He felt it brought out some of the actual flaws of capitalism. He really enjoyed it and gave it a 7.
Scott said that while the author “is only a liberal reformer”, the book is well worthwhile
because of the exposures it presents about yet another way in which capitalist society is
becoming more and more unfair and economically polarized. For that reason he has raised his rating
up from a 5 to a 6. Scott’s biggest beef with the book is that O’Neil provides only false solutions
to the ever-expanding use of the biased and unfair algorithms that she documents. She says
that those who create and use these biased and unfair algorithms should be alert to that unfairness
and avoid it. But what is to make them do so, especially when it goes against the overriding need to
enhance corporate profits? She says that the government should regulate this sort of thing.
But how can the government be brought to do any really effective regulation when the corporations
control the government? (Remember: America has the best politicians money can buy.) She says
that “we” (the public?) should “take steps to stop” these “weapons of math destruction”, as she
calls them (p. 203), but what steps exactly are really at all possible under this capitalist system?
Scott claims that “liberals are utopians”; that is, that they put forward only reformist “solutions”
which cannot actually be implemented.
Because O’Neil can only think in terms of the perpetual continuation of the current capitalist system, and because even within that distorted perspective she can only focus on her narrow understanding of the present situation in bourgeois society, she is unable to recognize that some of the things she talks about will become at least partially moot in the relatively near future. For example, she justifiably worries about unfair hiring algorithms which discriminate against minorities and the poor right now. But she ignores the even bigger fact that (because of automation and AI) many millions of jobs are disappearing quite rapidly in historical terms. Reforms of the sort that she is pushing are more feasible (if still inevitably only quite limited and temporary even then) during capitalist boom periods when the economy is expanding and good new jobs are rapidly becoming available. In the present period of prolonged economic crisis for U.S. and world capitalism “negative reforms” are far more common—that is, attacks on the working class which drive them down further, and which involves ever more unfairness and growing inequality in the process. (See neoliberalism and the necessity during economic crises for more labor_market_flexibility.) In other words, O’Neil is oblivious to the real overall social and economic situation and doesn’t place her discussion in that appropriate context.
Our guest Abbie felt that O’Neil’s book is good at describing the general problem with biased and unfair computer algorithms which make so many social decisions these days, and that she has some real familiarity with the material. But he also felt that she herself doesn’t fully understand everything she talks about and is even infected to some degree with a lot of the same sort of thinking that leads to the creation and use of such biased algorithms. In talking about hiring and promotions within companies, for example, she tends to see nothing wrong with some individuals rising above others, making it big and becoming the multi-millionaire bosses, as long as this is not done in clearly social discriminatory ways (by race, gender and so forth). So this is also still an individualist bourgeois perspective, but without the overt racism and sexism. Individuals struggling for their own special advantage, supremacy and privilege is still disgusting, even when racism and sexism do not play a role in it. And therefore, even then, algorithms which select the individuals who will succeed over all the rest who are destined to remain below them and miserable are still not really good or admirable things. Abbie also rated the book a 6.
Barbara found the book very revealing. She said that the breaking down of people and discrimination against them was really disappointing, as were the tendencies towards downward spirals for people. It presents a picture of the destruction of generation, she said. She gave the book a 5.
Kevin agreed a lot with what Kirby said. But added that “the book reveals we are stuck in amber”. He pointed out that AI is definitely accelerating the problems O’Neal discusses. (“AI is dark-on-dark” he said.) Kevin said that her solutions are at best ineffective—if not Pollyannaish. “The power center will not allow her proposed fixes to go into effect.” It was an easy read, though there were jarring transitions at times. Kevin gave it a 7.
John hadn’t finished the book at the time of the meeting, but was somewhat surprised to find that he really does like it. “The inherent bias bult into the algorithms are pretty bad!” John thinks it is important to read this book, and rates it a 7. “It’s an enjoyable read”, he says.
Vicki gave the book an 8. Even though she is in this very computer field herself she said that “there was a lot that was new to me”. She found it quite interesting that switching to an analysis in terms of individual cohorts (dividing the data into separate groups) can affect the conclusions drawn in a major way. Vicki somewhat disagrees with Kirby’s criticism of the book. She agrees that O’Neil’s suggested solutions to these biased algorithm problems is extremely weak. Vicki pointed out that those who design algorithms often have little knowledge or insight as to how they will be used against people.
Vicki also gave us another example which she had heard of herself. A guy was fired from his job by a computer, but no human at the work place could figure out or say why. “The algorithm did it.”
Return to our complete list of books.
Return to our Science Group home page.