As I wrote back in May, Nate Silver is obviously a smart and energetic blogger, but he just isn’t a social scientist. That’s why it’s frustrating when his quickie statistical analyses draw more attention than the relevant political science scholarship (of which Silver frequently seems unaware).

For instance, Silver published an analysis today that claims to measure “the impact of lobbying by the insurance industry on the prospects for health care reform.” His statistical model predicts senators’ support for the so-called “public option” using their estimated DW-NOMINATE ideal points (a measure of their policy views), health costs in their state, and their total campaign contributions from health insurance and HMO PACs. He finds that senators who receive larger industry contributions are less likely to support the so-called public option. Silver extrapolates from his model to a world in which special interest contributions are banned and concludes that “the insurance industry’s influence appears to swing about nine votes against the public option.”

However, as GW’s John Sides notes on The Monkey Cage, “there is a potential endogeneity problem in the model that Silver constructs. Does health insurance PAC money make senators less likely to support the public option, or are Senators who tend to oppose or at least be skeptical about government health care programs more likely to get contributions from health insurers and HMOs?” In other words, we can’t be confident that contributions are causing senators to oppose the public option. Silver acknowledges this caveat in a parenthetical, but then concludes his article by making the causal claim that “the money is why… Democrats are facing an uphill battle on the issue.”

In addition, we can’t say much of anything about a world in which health industry contributions aren’t made. It’s an unobserved counterfactual. Silver’s estimate of nine votes being swung is essentially meaningless in its current form — the model is extrapolating far beyond the available data.

What’s so frustrating about Silver’s post, which was (disappointingly) praised by Paul Krugman on his New York Times blog, is that there is an extensive literature on this subject by political scientists and economists. Over the last 30+ years, these studies* have typically found minimal effects of campaign contributions on roll call votes in Congress even when scholars use more sophisticated techniques to address the causal inference problems described above (which are widely understood at this point). Given these prior findings and the inherent problems with Silver’s analytical approach, we should be extremely cautious about putting any weight on his conclusions.

(* See, for instance, pp. 112-117 of Ansolabehere, de Figueiredo, and Snyder 2003 [PDF] from the Journal of Economic Perspectives.)