Don't Listen to Nate Silver, Listen to Me
Lots of smart people are anxiously sharing links to Nate Silver’s 2016 Election Forecast on their social media profiles, largely because, one assumes, they are beginning to realize the implications of the fact that there is an actual presidential election happening and Donald Trump is an actual presidential candidate.
Is there a chance that Donald Trump could be the next president of the United States? Yes. There is a chance. It is well within the realm of possibility, and should that possibility come to pass, no one should be surprised. (Disappointed, yes, but not surprised.) A Trump presidency, however, remains, by Silver’s own accounting, pretty unlikely: taking into account “polls, the economy, and historical data,” Silver is currently predicting that Hillary Clinton has a 59.5 percent chance of winning, and Trump has a 40.5 percent chance of winning.
That is not the analysis that has everyone in a tizzy, however. Instead, they are linking to the “Now-cast,” which purports to show “Who would win an election today.” Currently, according to the Now-cast, Trump has a 53.8 percent chance of winning. (53.8! 538! Look at that.) Given that polls taken when one candidate has just had a convention and the other has not are largely understood to be meaningless—an understanding which Nate Silver surely shares—the only purpose of publishing data like this is to make people panic. Whatever the value of the larger FiveThirtyEight.com exercise (that is, of so-called data journalism), presenting this kind of information without any context merely provides readers with an opportunity to project their fantasies, fears, and anxieties onto a set of vaguely significant numbers.
Don't think people are really grasping how plausible it is that Trump could become president. It's a close election right now.
— Nate Silver (@NateSilver538) July 23, 2016
“Using numbers is neither necessary nor sufficient to produce good works of journalism,” Silver wrote in a blog post titled “What the Fox Knows,” introducing readers to his new site, FiveThirtyEight.com. (538's logo is a fox: a reference to a phrase attributed to the Greek poet Archilochus: “The fox knows many things, but the hedgehog knows one big thing.”) The post introduces old readers to the new site—Silver had been tempted away from the New York Times, where he had a blog, by ESPN—as well as new readers who were likely enticed by tales of his magic formulae that so accurately predicted the outcome of the 2012 presidential election.
As it turns out, Silver is actually quite disdainful of (most) attempts at prediction—at least in journalism and the social sciences. This is why his charts and data are presented as descriptions of how likely a given event is: It’s a way of hedging. Silver wrote:
No matter how well you understand a discrete event, it can be difficult to tell how much of it was unique to the circumstances, and how many of its lessons are generalizable into principles. But data journalism at least has some coherent methods of generalization. They are borrowed from the scientific method.
He went on:
As my book describes, predictions in the sciences (especially the social sciences) are often fairly poor. They usually get better after repeated trials and iterations. But they require a lot of work. One of our sports journalists, Benjamin Morris, suggests that you have almost no hope of beating Las Vegas unless you’ve spent at least 100 hours studying the betting line in question. I can imagine a few exceptions, but it’s a wise rule of thumb.
In conventional journalism, predictions are often treated as a parlor game, involving little effort and less accountability. (A variety of studies on the predictions made by McLaughlin Group panelists, for instance, find that they are no more accurate than random guesses.) Predictions are usually outsourced to opinion journalists, who may have less subject-matter knowledge than reporters do.
To reiterate: It’s tough to make predictions, especially about the future. But one of the potential advantages of data journalism is that it generalizes better than traditional approaches, particularly as data sets increase in scale to become larger and more complex.
And yet that data journalism failed quite spectacularly to anticipate the rise of the nativist reality television demagogue who currently leads the Republican Party. So, in May, Silver penned a lengthy mea culpa in which he offered a grandiose non-apology for how he “Acted Like A Pundit And Screwed Up On Donald Trump.” (“Falsifiability is one of the big reasons we make predictions,” he notes in an early parenthetical. Sure it is!)
Looking ahead to the next “Trump-like candidate” who enters the field in 2020 or 2024, Silver asked, “might the conventional wisdom overcompensate and overrate his chances?” Maybe! But maybe not. Also: “The Republican Party’s immune system will be on high alert against future Trumps,” he writes. Or maybe the Republican Party is changed forever!
Cases like these are why you should be wary about claims that journalists (data-driven or otherwise) ought to have known better. Very often, it’s hindsight bias, sometimes mixed with cherry-picking and — since a lot of people got Trump wrong — occasionally a pinch of hypocrisy.
Still, it’s probably helpful to have a case like Trump in our collective memories. It’s a reminder that we live in an uncertain world and that both rigor and humility are needed when trying to make sense of it.
“We live in an uncertain world.” Damn, that’s rigorous as hell.
@fierman: Polls-plus is the most accurate, historically. But the now-cast is fun if you like to panic!
— Nate Silver (@NateSilver538) July 25, 2016
Before the Republican convention, Donald Trump had zero popular votes and zero electoral votes and Hillary Clinton had zero popular votes and zero electoral votes. This morning, Donald Trump had zero popular votes and zero electoral votes and Hillary Clinton had zero popular votes and zero electoral votes. So, what does the “Now-cast” tell us, really? “Who would win an election today,” it says. Well, maybe. Really what it tells us is who would win an election today if the election were just polls—a weird, intellectually corrupt meta-activity that is treated as if it’s the genuine activity because the genuine activity has not happened yet. It is highly misleading and pointless to frame that data in terms of a daily running update about Who Would Win if the Election Were Held Today.
Well, okay. So what? People like to read this kind of thing. It’s entertaining: They like to feel that the election is changing moment to moment. But it’s also antithetical to the Nate Silver appeal, which is telling us what’s going to happen on Election Day. Telling us what would happen today is not the point! Ostensibly, Silver’s goal is making his best prediction for the real event, not humping every fluctuation. It is also a little silly that Silver spent almost the entire primary trying to convince the media and the general populace that Trump was not a general election threat. And here they both are, scaring people.