Two of the nation’s best pollsters have recently weighed in on the state of the Iowa Democratic race, with very different results. One subtle methodological choice might explain part of the difference, and on balance there is reason to think Bernie Sanders might be a bit stronger in one of the two polls than the topline result suggested.
The first Iowa survey was the one conducted by Ann Selzer and sponsored by The Des Moines Register and CNN. On Friday, it found Mr. Sanders with a three-point lead and 20 percent of the vote. Elizabeth Warren and Pete Buttigieg were at 17 percent and 16 percent. Joe Biden was in fourth, with 15 percent.
On Monday the other pollster, Monmouth, found Mr. Biden in first with 24 percent of the vote — nine points higher than in the Selzer poll. Mr. Sanders was in second, with 18 percent, while Mr. Buttigieg and Ms. Warren landed at 17 percent and 15 percent.
Either way, the race is close. Any of the four leading candidates could prevail, given the long history of late movement in Iowa caucus polling. In general, modest disagreements between polls ought to be expected, and it is no surprise that polls would frequently show different winners in a close four-way race like this one.
But the difference between the two poll results is potentially very meaningful for campaigns, journalists and voters, who will make important choices heading into the final stretch based on whether they believe certain candidates are strong or weak. It could easily make the difference in the debate tomorrow night, for instance, on whether a candidate should mainly focus on attacking Mr. Biden or Mr. Sanders. It could make the difference between whether an editor assigns an article on Mr. Sanders’s “Medicare for All” plan, for example, or on Mr. Biden’s support for the “public option.”
And although the difference between the two polls isn’t necessarily huge from a statistical perspective, the nine-point difference in Mr. Biden’s support and the net 11-point difference in the Sanders-versus-Biden margin does push the limits of what can be easily attributed to sampling error alone.
One part of the explanation is the sampling frame of the survey, or the people who could be selected to participate in the poll.
According to the Monmouth poll’s methodology description, the poll sample was drawn from “a list of registered Democratic and unaffiliated voters who voted in at least one of the last two state primary elections or the 2018 general election or have registered to vote since November 2018.”
The Selzer poll, on the other hand, includes all active registered voters in Iowa, which would allow a few groups who are excluded from the Monmouth poll: registered Republicans who say they’ll participate in the Democratic caucus, or voters who registered before 2018 and did not participate in either of the last two primaries or the 2018 general election. (The Selzer survey is traditionally the most eagerly anticipated poll of the Iowa cycle.)
The group of voters that the Monmouth poll missed is not necessarily a large part of the Iowa caucus electorate. In The New York Times/Siena College poll in October, which surveyed all registered voters, 92 percent of the voters who said they would “probably” participate in the Iowa Democratic caucus would have been included in the Monmouth poll. And this group of 8 percent includes one subgroup of voters — registered Republicans — who aren’t necessarily favorable to Mr. Sanders.
But on balance, the 8 percent of probable Iowa caucusgoers left out of the Monmouth poll are quite different from other voters. They are younger; less likely to have graduated from college; likelier to be men; and likelier to support Mr. Sanders.
Over all, 33 percent of the respondents who would have been excluded in the Monmouth poll backed Mr. Sanders in the Times/Siena poll from October, and this was at a point in the race when Mr. Sanders was weaker in the polls than he is today. (He gained five points in Monday’s Monmouth poll compared with the one taken in early November, just after the Times/Siena poll.)
Mr. Sanders was at 38 percent support among the registered Democrats or independents excluded by the Monmouth vote history screen.
How much would this matter in practice? In the Times/Siena poll from October, excluding these voters would have cost Mr. Sanders two percentage points — and that would have expanded to three points had we reassigned the support of voters who dropped out of the race to their second choice. Mr. Biden’s standing would have improved by a point, yielding a net four-point shift in Mr. Biden’s favor, compared with Mr. Sanders, by excluding these voters.
This doesn’t cover the whole gap between the Monmouth poll and the Selzer poll — only about a third of it, and mainly explains Mr. Sanders’s diminished support, not Mr. Biden’s strength.
Some of Mr. Biden’s strength in the Monmouth poll, compared with the Selzer poll, might be attributed to the absence of education weights in the Selzer poll, which would tend to yield a much better educated and therefore less Biden-friendly electorate. Here, the Monmouth poll is on firmer ground; failing to weight by education is regarded as a major reason polls underestimated Donald J. Trump in 2016. And this would also help explain Ms. Warren’s relative strength in the Selzer poll, as she excels among college-educated voters.
On the other hand, the split in Mr. Biden’s support by education in the Monmouth poll — 28 percent among voters without a college degree, and 21 percent among those with a degree — is not so large that one would expect education weighting to affect the result by much more than a point.
Not every difference between two polls can be attributed to a clear methodological choice. Random chance plays a big role, especially in smaller samples like these. But here, the sampling frame makes a clear difference for a candidate (Mr. Sanders) who is clearly, if only modestly, disadvantaged.