What's up with Mass. Senate Polls?...

Thank you Hotair.com

The new Boston Globe poll showing Democrat Martha Coakley beating Republican Scott Brown by 15-points in the race for Ted Kennedy’s Senate seat, when contrasted with Public Policy Polling showing a dead heat, has people scratching their heads. So what’s up with that?

TPM’s Josh Marshall claims it is “all about the screen,” referring to how each poll selected likely voters. The two polls do appear to screen differently — the Globe specifically asked, while PPP selects voters in recent elections (Personally, I prefer the latter). But Marshall is almost certainly wrong that it’s all about the screen. AmSpec’s Philip Klein does a much better job of outlining differences between the two polls: (1) The Globe sample is much more Democratic; (2) PPP has Brown winning Independents by a huge margin; (3) PPP polled after the Globe. But the differences do not stop there. The Globe poll used a smaller sample, producing a larger margin of error. The Globe poll used live interviewers, while PPP uses automated calling. The sequencing of key questions also differs. For example, PPP started by asking for whom the respondent would vote, while the Globe asked about party registration and candidate favorability first. The Globe specifically included Independent candidate Joe Kennedy, while PPP did not (though one might have expected a faux Kennedy to hurt Coakley, that turned out not to be the case).

The import of the first two factors Klein identified is crystallized in his observation that the Globe only polled 83 independents, while PPP polled about 290. The Globe asked specifically in terms of registration, while PPP did not, which may account for some of the gap. Even so, there will be a much larger margin of error in a sub-sample as small as 83. These issues also filter down to the different results on candidate favorability in both polls — the Globe speaks of Coakley’s “durable statewide popularity,” which you do not see in the PPP results.

One point of agreement between the two polls is the role of relative intensity. PPP reports that “66% of GOP voters say they are ‘very excited’ about casting their votes, while only 48% of Democrats express that sentiment.” The Globe reports that “Brown matches Coakley – both were at 47 percent – among the roughly 1 in 4 respondents who said they were ‘extremely interested’’ in the race.” Some might be tempted to frame these numbers as an “enthusiasm gap,” though it is probably more accurate to note that the Right tends to vote more regularly than the Left, and that the key for Coakley will be turning out enough of the state’s much larger pool of Democrats.

Finally, I should add a note about PPP. The news accounts (and blog commenters in stories involving PPP) almost always note that the Democratic firm infamously showed Conservative Doug Hoffman leading Democrat Bill Owens by 17 points in its final poll of the NY-23 Congressional special election last November (Owens won by 4 points). There were a number of practical problems with polling that particular election, though I do not think they fully explain PPP’s call. Rather, everyone should remember that the margin of error reported by most polls assumes a 95% confidence interval. That means that given repeated samples, 19 of 20 would produce results for any given question falling within the stated margin of error. PPP’s call in NY-23 was likely that bad outlier for which the risk always exists, even with the best of pollsters. That is why it is always better to have more than one poll to examine, even if the results are maddening when we have only two to examine.

This post was promoted from GreenRoom to HotAir.com.
To see the comments on the original post, look here.