Hyping the gas price-Bush approval link

Matthew Yglesias flags netroots activist Chris Bowers hyping a graph plotting gas prices (inverse) against presidential approval:

It isn't discussed often enough, but Professor Pollkatz regularly produces updates one of the most important statistical projections for American politics, ever:

A round of semi-apocalyptic pontification then ensues:

Our move off the petroleum economy is still slow, and will even take decades under an extremely aggressive Apollo style renewable energy program. So, in short, there is no reason to expect that future administrations will have approval ratings dramatically better than those currently enjoyed by both Bush and Congress. Ending the war will help, and providing millions of people with health care will help, too. However, gas prices will still remain high, making approval ratings in the 40s and 50s about the best anyone can do even under the most favorable conditions until at least 2020.

The days of super popular administrations, ala the fourth, fifth and sixth years of Reagan, and like the final five years of Clinton, are probably over for the foreseeable future...

Really? No more popular administrations? I'd be happy to take that wager.

More fundamentally, as Yglesias notes, this is a classic example of the fallacy in which people see a correlation and infer a causal relationship:

I was super-impressed the first time I saw that chart, but now I'm not so sure. Aside from the fact that gasoline has generally gotten more expensive and Bush generally gotten less popular, are we really seeing a correlation here? There must be any number of quantities that have also generally moved in one direction during the relevant time period.

Consider the quality of the basketball teams fielded by the New Jersey Nets. Like Bush, they were at their best in the season immediately following 9/11 and have been in slow but steady decline since then. But the Nets aren't exercising a causal influence on Bush's popularity. What's more, I think the price of gas is being demarcated in nominal terms here, which is clearly the wrong way to do it.

Yglesias is right -- since 9/11, any quantity that has been trending in one direction will be correlated with President Bush's approval (this is elementary time series statistics). And if you look at the immediate period after 9/11 period, you see that Bush's approval shot up before the price of gas declined, which makes no sense if you think gas is driving his approval ratings. Finally, there is no obvious relationship between the two before 9/11.

Does this mean the price of gas doesn't affect approval? No. But this chart greatly exaggerates the relationship.

Update 10/8 4:34 PM: I forgot I debunked this argument much more thoroughly last year:

When we estimate a quick-and-dirty model for presidential approval from January 2001-June 2006 using a lag of approval in the previous month (which corrects the serial correlation problem), a variable capturing the 9/11 approval boost in Sept./Oct. 2001, a variable for the Iraq invasion in March 2003, average hourly wages, total payroll employment, and the logarithm of inflation-adjusted gas prices, we find that the effect of gas prices is negative but not quite statistically significant at conventional levels. 9/11 and the Iraq invasion, by contrast, have highly significant effects. (A more complex model that takes account of the long, slow decline in the 9/11 boost would likely wipe out the gas effect completely.)

Now it's certainly plausible that gasoline prices have some effect on approval, but not to the extent that Harwood or Page suggest. Given the weak relationships that existed during past administrations (which Harwood acknowledges), the relationship that we observe during Bush's presidency seems likely to be a statistical artifact. It is very premature to call gas prices the "strongest factor" affecting approval. (For those who are interested, The Macro Polity is to my mind the definitive political science work on the factors influencing presidential approval.)

Update 9/22 10:47 AM: Using sophisticated techniques and better data than my quick-and-dirty model above, the distinguished political scientists Nathaniel Beck, Simon Jackman, and Howard Rosenthal report results for the determinants of Bush approval that mirror my analysis above (PDF):

Immediately note that the approval series is dominated by the long, almost uninterrupted decline after the peak in the 9/11 aftermath. Hence, any variable that trends in a similar way will emerge as a good predictor of approval, at least in these data. Figures 9 and 10 plot the relationship between weekly gasoline prices and approval and cumulative U.S. deaths in Iraq, and presidential approval, respectively. The latter variable trends up, by construction, and gas prices also generally trend up over the post 9/11 phase of Bush’s presidency. We entered these variables, plus the log of first time unemployment claims in an augmented version of our transition model... The coefficient on the covariates are something of a mixed bag, with the estimated effects on changes in gas prices not unambiguously signed at conventional levels of statistical significance (i.e., the posterior probability that changes in gas prices drive approval down is just .77). Cumulative U.S. deaths in Iraq appears to drive approval down (again, if for no other reason than both variables trend in the same direction over much of the time series). Similarly, first time unemployment claims generally trend down over the course of the Bush presidency, around seasonal variation, and so picks up an unambiguously positive coefficient.

Comments

[Richard Weaver]

You are right that the similarity of the trend over a period of several years is not significant, and I would hope that few readers would take it seriously. But I also imagine that I can see in this (and a proper statistical study could be done to confirm or contradict this) that there ARE correlations in the short term. These are the more interesting.

[fuel additives]

Honestly, that graph doesn't make a lot of sense to me. You could tie so many things together in the same way, but that doesn't mean its true.

[Shinobi]

Statistically I can make a graph of any number of unrelated things and make them look related. As I told my stats 101 students over and over Correlation does not mean Causation.