How election polling is like market research – is like football?
I think I speak for just about everyone I know in saying I’m tired of rehashing the election results. We were all surprised by the outcome, and as we all know, the pollsters had it wrong. The odds of a Trump victory going into election day were calculated to be about the same as an NFL place kicker missing a 37-yard field goal. Many asked, how could these professional pollsters with advanced sampling techniques and analytics have it so wrong?
Well, they weren’t so wrong. The error rate in 2016 was only slightly higher than that of 2 of the last 6 presidential elections (3.4 vs. 3.6%), per the polling analyst group FiveThirtyEight, as reported by the New York Times. The point here is not to pontificate about the election, nor is it to level charges of malpractice on the part of political pollsters.
The field of political polling is closer than a first-cousin to marketing research; the two are more like step-siblings. One common element is that the issues with sample bias are strikingly similar. Even stellar back-end analytics can’t make up for a badly biased sample. As we saw in the pre-election polling, it’s hard to compensate for the lack of white, working class males showing up in a sample from Wisconsin or Michigan. Similarly, it’s quite difficult to “model-out” the sample bias in a market research study.
So what’s the upshot of all this? Does it mean research shouldn’t be done?
Quite the contrary; it’s just that over-reliance on data and analytics without an equivalent understanding of the story underlying the data can lead to directionally wrong results. I can’t say how political pollsters can do a better job than they do. However, market researchers need to be mindful of this lesson, borrowed from politics: always seek to strike a balance between quantitative and qualitative methods to get to the compelling story that lies within.
You can miss the field goal, but still win the game.