The entire polling industry faced an existential crisis on the grim morning of November 9, 2016. The morning before, on Election Day, nearly every mainstream media outlet, from the New York Times to the Princeton Election Consortium, predicted Mrs. Clinton’s change of winning from 70 to 99 percent. Democrats went to the polls with a sigh of relief. Cakes were baked, firecrackers were bought, and media sites tailored their content for a Hillary win.
Then Trump won in an Electoral College landslide. The polls had failed miserably, but this wasn’t the first time in 2016 that they had confidently predicted the exact opposite of the result. Just a few months earlier, England had voted to Brexit, against all odds and polls. The industry should have taken stock then.
“It’s the overselling of precision,” said Dr. Pradeep Mutalik, a scientist at the Yale Center for Medical Informatics, who had predicted that some of the polls could be off by 15 to 20 percent prior to the election. If polling is done correctly, using a true representative sample of the population, then the results should always be within the margin of error. But reputable sources had placed Mr. Trump below that margin —a sign that the poll’s fundamental assumptions were wrong.
It will take months, even years, for the industry to re-align and come up with an accurate model for the 21st century—if, in the age where young people are more likely to express opinions on Twitter than sit through a ten-minute phone call with a random pollster, the “polling industry” as we know it can even survive. It would be smart for them to take heed of these three lessons.
1. Watch for confirmation bias
The media isn’t totally dominated by liberals, but it is dominated by elites, who have watched their handpicked presidential candidates, on the left and right, ascend the presidency since World War II. The party nominating processes are traditionally a time for corporations, party elites, and others to whittle out the populists, revolutionaries, and true progressives—for example, Bernie Sanders this year. The Republican establishment would have done the same to Trump if didn’t have his own deep coffers to draw from.
The media, which had painted (for not unjustifiable reasons) Trump as a sexist, racist, buffoon, could not fathom a world or America in which he became President. To do so would challenge their basic assumptions about America, the world we live in, and humanity itself. Their subconscious desires affected their presentation of the data.
After all, there were signs on the morning of the Election that Clinton was in serious trouble in the Midwest. But the Democratic Party ignored those in favor of the outcome it expected—that the Midwest, for decades a bastion of working-class Democratic voters and a part of the Obama coalition, would stay that way. They were caught by surprise because of their confirmation bias—the tendency to interpret new evidence as confirmation of one’s existing beliefs and theories.
Computers can collect and analyze data, but its up to humans to interpret it. If the pollsters and media, many of whom are based on the coasts, away from the America that voted for Trump—they would have seen that a 1-2% percent lead in states Democrats traditionally won by landslides was no lead to feel good about, and rather a cause for extreme concern that was never addressed. It was Trump who spent the last hours of the campaign in Michigan, a state he won by 10,704 votes.
2. Current analysis tools are outdated
Most pollsters currently use the same methods of decades ago—calling phones, assembling focus groups, etc. Clearly in retrospect, there are certain groups that those methods don’t reach. Social media is this generation’s media—and so “social listening” techniques on sites like Reddit and Twitter can serve as a more effective gauge of a population’s mood.
Additionally, a complete transition out of landlines is needed (landlines are heavily skewed towards older people), as well as movements into web and digital polling, supplemented by creative programming to engage internet denizens.
3. Prepare for the human factor
What won the election wasn’t the the rabid Trump fan base or the Clinton coalition. What gave Trump the victory were the last few percentage points of undecided voters, who broke at the last second largely in favor of Trump. In a change election, more undecided voters will, at the last second, vote in favor of change, which was Trump.
How the polling industry incorporates this is important. The world may seem easy to figure out with enough big data. But there is always a human element that, well, only humans can account for.
The human element is what enabled Dilbert author and hypnotist Scott Adams to correctly call the election for Trump a year in advance. Regardless of how he was doing in the polls, Adams always factored in the persuasion element. Pollsters can only go so far when given an “undecided” response. It is psychological insight that predicts where the undecideds go, and that predicts election.
