Margin of error alert: The Pelosi/Cheney poll

Drudge, ABC's Jonathan Karl (guest-writing Mike Allen's Playbook at Politico), FoxNews.com, and National Review's Jim Geraghty all claim that a new Gallup poll shows that Dick Cheney is more popular than Nancy Pelosi:

DRUDGE: "GALLUP: Cheney more popular than Pelosi!"

KARL: "Liz will like this: Gallup puts former Vice President Cheney’s popularity slightly above Nancy Pelosi’s."

FOXNEWS.COM: "A new Gallup poll out Friday showed Pelosi's favorability ratings are lower than that of former Vice President Dick Cheney, a popular target for Bush administration critics."

GERAGHTY: "Indeed, Nancy Pelosi is less popular than Dick Cheney."

Here are the actual results of the poll in question:

While Gallup does estimate that 37% of Americans have a favorable impression of Dick Cheney and 34% have a favorable impression of Nancy Pelosi, the poll has a 3% margin of error, which means we can't be confident that the difference between those numbers is statistically significant. In addition, the poll also estimates that Cheney has somewhat higher unfavorable ratings than Pelosi (though, again, we can't be confident that the difference is statistically significant). If we take Pelosi and Cheney's unfavorable ratings into account as well, the ratio of favorable to unfavorable ratings are almost identical -- 34/50 = .680 for Pelosi, 37/54 = .685 for Cheney.

Sadly, these journalists are making a common mistake -- the quantitative illiteracy of the press means that minor differences in poll numbers are often hyped without regard for the margin of error. Can't ABC hire an undergrad stats major to be an intern or something?

Update 6/5 11:08 PM: Jennifer Rubin at Commentary Magazine did the same thing -- the headline of her post is "Cheney Is More Popular Than Pelosi." No!

Comments

[David]

If the reporters had simply cribbed from Gallop's accurate write-up, they would have avoided the error pointed out by Brendan.

In fact, the total uncertainty is even greater than the figure reported as "margin of error." That figure represents sampling error. As Gallop points out, "In addition to sampling error, question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of public opinion polls."

[Jinchi]

Can't ABC hire an undergrad stats major to be an intern or something?

Sure they could, but that would have just killed an easy story. Why would they want to do that?