Yes, polls. Again. Why am I still talking about them?
Well, I see a lot of statements on the subject, here and elsewhere, that are just plain incorrect. That’s not new, either; it’s been going on as long as I remember. And not just concerning the recent election (the one I’ll mainly be discussing here); about past ones, too. So I think it’s good to point out a few salient facts—and then everyone can keep on arguing about the value of polls, the methodology of polls, and the intentions of pollsters.
To begin with, it’s necessary to understand that polling is inherently difficult to get right. That makes it flawed as a predictor (as I often say) but not totally flawed by any means. And besides, it’s the best predictor we’ve got. Crowds don’t matter, nor do yard signs (as I explained here, and as Romney discovered in 2012).
But if one pollster keeps getting it wrong when all the other pollsters get it right, that pollster isn’t going to last long, or if they last for a while they won’t have a good reputation. Pollsters’ most financially productive money-making activities—internal polling for candidates, and market research for businesses—depend on maintaining their reputation for accuracy, so they are interested in getting it right. There are also organizations that rank pollsters for accuracy over time, so among people who hire pollsters (for example, for market research, which is lucrative), the pollsters’ reputations are known.
But what if most of the pollsters get it wrong? Or did they?
I have yet to understand why so many people seem to think that the polls were so very, outlandishly wrong this year (except at the state level in a couple of states, particularly Wisconsin; but state polls are known to be much more difficult to do in terms of sampling, and are often somewhat outdated by the time the election rolls around). Why do I say the national polls were not so far off? Take a look at what the polls were actually saying the day before the election, rather than what you think you remember about what they were saying.
Remember also, when you look at this article (written one day before the election), that Hillary Clinton (so far, anyway) has won the popular vote by 1.27%. So any poll that showed her winning the popular vote by around that figure would have been remarkably accurate:
With just one day to go until the 2016 presidential election, Hillary Clinton’s support is at 46.8% while Donald Trump’s is at 44.3%.
When analysts refer to the Democrat’s 2.5 percentage point lead, they are talking about the difference between those two figures in polling averages.
It’s an important number but it’s also probably an inaccurate one. That’s because polling averages do not capture fully a few factors that will affect the final outcome.
So, the average of polls right before the election was only off from the actual result by 1.23%, and they got the popular vote winner (which is what national polls measure) right. That’s really pretty close.
The article goes on to state some of those factors that are not reflected in the polls but which affect the final outcome—early voters, turnout, margin of error, and vote distribution and the Electoral College (this turned out to be a very large factor which determined Trump as the winner). The article also mentions an unusual amount of uncertainty for this particular election. It doesn’t mention the undecideds, but we have learned since the election that they broke disproportionately for Trump, and this may indeed have made the difference between his winning and losing.
So I actually don’t see that the polls were especially inaccurate.
Take a look at what Nate Silver was saying two days before the election (Silver isn’t a pollster, but he analyzes polls and makes predictions based on them, and he has a pretty good track record):
In an appearance on ABC’s “This Week,” the FiveThirtyEight chief claimed Clinton is a “2-to-1 favorite,” but noted that recent polls show Republican presidential nominee Donald Trump with a slight edge in electoral college-heavy states like Florida, Ohio, and North Carolina.
“The electoral college math is actually less solid for Clinton than it was for Obama four years ago, where four years ago we had Obama ahead 320-some electoral votes. Clinton has about 270,” Silver said.
“So she’s one state away from potentially losing the electoral college. You’d rather be in her shoes than Donald Trump’s, but it’s not a terribly safe position.”
Finally, take a look at Nate Silver’s list of polls taken right before the election, from an article written two days before the election. Fourteen polls are listed. Two show Trump leading (by +1, and by +8). One shows a tie. Four more show Clinton leading by either 1% or 2%—and since it turns out she actually won the popular vote by 1.27% (at the time of this writing), I’d say those particular polls were highly accurate, the most accurate of all. That means that seven out of fourteen polls either got the winner right (Trump) or got the actual vote right or extremely close to right. That sounds accurate to me for six of the polls, which is approximately half (one of those that got the winner right was very inaccurate, so I’m not adding that to the “accurate” list, although it was right about the Trump win).
What’s more, three more of the polls measured Clinton at +3 in the popular vote. That turned out to be only 1.73% off. Not too shabby, either. That leaves two polls that said Clinton +4, and two that said Clinton +5. So the majority (9 polls) got it right or came quite close. And the LA Times poll, one of the two that got the winner right, was also the most inaccurate poll, because it showed Trump ahead by +8, which was actually 9.27 points off, whereas many of the polls that got the winner wrong actually were quite accurate about the popular vote.
Polling is flawed, as I’ve said almost every time I write about it. It’s just a guide. In races that aren’t close, it can be a very good guide. In close races (and this was definitely a close race) polls are iffy. But, contradictory to many people’s recollections, most of the polls this year right before the election predicted a close race. And if most also predicted a Clinton victory in the popular vote—well, they turned out to have been correct about that, too.
Here’s what Silver wrote two days before the election, and remember again that in the actual election Clinton seems to have won by about 1.27% (and that figure could actually rise over time, and has been steadily doing so):
On average across the 14 polls, Clinton’s lead is 1.9 percentage points…Or you can take the median instead of the average, which is also 2.5 points. Use only polls rated A-minus or higher? Her lead is 2.6 points. Use live-telephone polls only? It’s 2.3 points. Use only polls used to determine eligibility for the presidential debates? That gets you to 3.5 points. Use only the very recent surveys, which conducted all of their interviews in November? Her lead averages 2.3 points in those.
Not so very far off. And after reading the following from that same November 6th article by Silver, you may agree with me that Silver (who works with the poll figures as a whole, analyzing and commenting on them) was surprisingly accurate:
How about polls of swing states in particular? Right now, the tipping-point state in our forecast ”” the state that would provide the decisive 270th electoral vote if the polls got things exactly right ”” is New Hampshire. There, Clinton leads by only 1.7 percentage points in our adjusted polling average, as several recent polls show Trump tied or slightly ahead, along with others that still give Clinton the lead. Thus, Clinton’s doing a little bit worse in the tipping-point state than she is overall ”” a sign that she might win the popular vote but lose the Electoral College.
In case you’re unaware of what happened in New Hampshire, Clinton won—47.62% to Trump’s 47.25%, a margin of 2687 votes out of a total of 694,307 cast. So the polls in New Hampshire seem to have been pretty good; they predicted both a very close race (true) and the eventual winner in that state.
I was very impressed with Nate Silver’s predictions in 2012, and I remain impressed. And the polls weren’t so shabby, either.



