« New at CJR: The misleading momentum narrative | Main | Knowledge does not guard against conspiracy theories »

October 30, 2012


1. Brendan admits that Silver's approach is a "black box statistical model" and "not...scientifically validated." Yet, he blasts those who criticizes that model. (In fairness to Brendan, he also implicitly criticizes those who give this model "talismanic quality".)

2. IMHO a poll is essentially a model predicting how the election would turn out if it were held during the sampling period. I call it a model because the surveys generally include various sampling and/or adjustment schemes. Adjustments are essential, since the response rate is so low. Simply contacting people at random and using raw results would likely introduce a bias.

So, which is more reliable -- Gallup showing Romney at +5 or an academic model showing an Obama victory? Brendan's a political scientist, so it's natural for him to prefer the political science studies. I'm not so sure. Gallup has been doing this for a long, long time....

3. As Brendan points out, the probabilistic nature of Silver's prediction means that a Romney victory would not prove that model is wrong. But, a theory that cannot be falsified isn't science.

In fact, if Silver or Gallup or any of the others miss badly, I'd expect them to modify their models and survey procedures. So, no matter what happens in 2012, in 2016 we'll be presented with new models and surveys that either succeeded in 2012 or that didn't fail in 2012. And, yet some of the 2016 models and surveys will get it wrong.

Great article! I remember a personal friend stating that Nate Silver got Question 1 in Maine (The gay marriage bill) wrong. I think Mr. Silver gave it around a 75% chance of failing and then it passed. He never claimed it WOULD pass he claimed 75% chance of pass.

Nate actually predicted the Maine vote would go exactly how it went. He gave his stats but in finale said it would lose by almost exactly what it lost by.

A Canadian's view of Silver's model at http://www2.macleans.ca/2012/11/04/tarnished-silver-assessing-the-new-king-of-stats/

The comments to this entry are closed.