Before the 2018 election, I said the best way to push back against those who call polls “fake news” was for the polls to predict the election results.
It turns out public polling passed this test with flying colors. Nonpartisan polls taken within three weeks of the 2018 election were far more accurate than the average poll since 1998.
The biggest test of 2018 was the battle for control of the House. There are two different ways to poll House races.
The most familiar is the generic congressional ballot, which usually asks voters to say whether they will vote for the Democratic or Republican candidates in their districts. From 1998 to 2016, the difference between the average generic ballot poll and the House national vote was 3.8 points. This year (with some votes still to be counted) it looks like it’s going to be only 2.8 points, which is a full point closer to the result than the average.
The other way to poll House races is by individual district. Those polls fared just as well. Nonpartisan House polls have historically missed the mark by an average of 5.9 points. This year it was just 4.9 points. Again, that means the average district poll was a full point closer to the result than usual.
That increase in accuracy was driven in large part by the Siena College/New York Times polls, whose surveys made up the bulk of district level polling and had an average absolute error of just about 3 points. That’s nearly 3 points better than average, which is off the charts good.
Statewide polling also had a strong year, although it should be noted that Senate and governors’ polling did pick fewer winners than usual.
The average poll in the Senate was off by only 4.2 points. The average Senate poll historically has been off by 5.2 points, which means this year’s polls were a point better than average. Likewise, the average governor’s poll had an error rate of 4.4 points. That’s 0.7 point more accurate than the average governor’s poll since 1998.
But there were also a number of cases where the polling had one candidate winning but the other candidate won the election. This year nonpartisan Senate polls called the correct winner 77% of the time, compared with 84% historically. Governor polls saw a nearly identical 76% of races called correctly, compared with 84% historically.
Why the difference? Much of it has to do with a lot of races forecast to be close this year. Over 20% of pre-election polls for the Senate and governor’s races had margins of 2 points or less. Historically that’s under 20%.