From political science professor Jacqueline Stevens (via Volokh):
It’s an open secret in my discipline: in terms of accurate political predictions (the field’s benchmark for what counts as science), my colleagues have failed spectacularly and wasted colossal amounts of time and money. The most obvious example may be political scientists’ insistence, during the cold war, that the Soviet Union would persist as a nuclear threat to the United States. In 1993, in the journal International Security, for example, the cold war historian John Lewis Gaddis wrote that the demise of the Soviet Union was “of such importance that no approach to the study of international relations claiming both foresight and competence should have failed to see it coming.” And yet, he noted, “None actually did so.”…
in the 1980s, the political psychologist Philip E. Tetlock began systematically quizzing 284 political experts ”” most of whom were political science Ph.D.’s ”” on dozens of basic questions, like whether a country would go to war, leave NATO or change its boundaries or a political leader would remain in office. His book “Expert Political Judgment: How Good Is It? How Can We Know?” won the A.P.S.A.’s prize for the best book published on government, politics or international affairs.
Professor Tetlock’s main finding? Chimps randomly throwing darts at the possible outcomes would have done almost as well as the experts.
Aha! In Part 5 of my change story, I wrote the following about the failure of experts to see the coming fall of the Soviet Union:
If the experts”“academic, governmental, and media”“had been unable to foresee this, then how could I trust them to guide me in the future? In retrospect, it was probably the first time I began to distrust my usual sources of information, although I certainly didn’t see them as lying”“I saw them as incompetent, really no better than bad fortunetellers.
What they seemed to lack was an overview, a sense of history and pattern. Newspapers could report on events, but those events seemed disconnected from each other: first this happened, then that happened, then the other thing happened, and then the next, and so on and so forth. In the titanic decades-long battle between the US and the USSR, there had been a certain underlying narrative (yes, sometimes that word is appropriate) that involved the threat of Armageddon, and the necessity to avoid it at almost all costs, while stopping the spread of Communism. Although T.S. Eliot had said the world would end “not with a bang but a whimper,” who ever thought the Soviet Union would end in such a whimpery way, and especially without much forewarning? It seemed preposterous, something like that moment in the Wizard of Oz when Dorothy throws the bucket of water on the Wicked Witch, who dissolves into a steaming heap of clothing, crying “I’m melting, melting.”
Although these events predated my change by over a decade, they sounded an early warning bell about experts that said “beware.”
And yet I don’t especially fault them, except when they demonstrate the hubris of thinking they can predict the future at all: accurate predictions are just too hard to make, as this review of Tetlock’s book makes clear. There are just too many variables and too much complexity.
Does this begin to sound familiar? AGW, anyone? But that’s science, and the hallmark of science is the ability to predict outcomes. If you mix this and that in certain proportions and subject them to heat, for example, you will reliably get a chemical reaction that produces another substance in a certain quantity. If Einstein’s relativity theory was correct, its prediction that “massive, spinning objects like Earth should warp space and time around them, as well as drag space and time along as they rotate” would be able to be confirmed, and that’s the case. And so on and so forth.
But the problem with climate science is that it would be fiendishly difficult to design an experiment that would test its accuracy. Climate science bears little resemblance to the sciences such as chemistry and physics described in the previous paragraph, and seems more akin to the system of prognostication described by Tetlock. So it’s strange that people feel they can rely on its predictive abilities.
Is anyone really good at consistently making predictions about complex events? I don’t think so. I think that what tends to be happening is that the list of people making predictions is long, and so somebody is bound to be right each time. It’s just not usually the same person twice—or three or four or five times, which would be even more impressive.




