Be careful interpreting primary results!

My posts analyzing state-level predictors of support for Barack Obama were cited by Richard Florida, the academic guru of the "creative class," in the Canadian newspaper The Globe and Mail:

Ms. Clinton is more popular among voters without college degrees. Meanwhile, Duke University political scientist Brendan Nyhan has crunched numbers that show a college education to be a big predictor for Obama support.

However, Florida's phrasing is potentially misleading -- my results show that average state education levels (i.e. percentage of adults with college degrees) appear to be positively associated with state-level support for Obama. We can't make any inferences about individual-level behavior from macro-level results (this is known as the ecological fallacy).

On a related note, Brad DeLong posts the following graphic from a David Sirota column in In These Times:

Sirota, with whom I've had my differences in the past, constructs the graph by plotting Obama's margin against Hillary Clinton excluding "the two senators’ home states (Illinois, New York and Arkansas), the two states where Edwards was a major factor (New Hampshire and Iowa) ... the one state where only Clinton was on the ballot (Michigan) and ... the four states whose Hispanic population is over 25 percent" (CA, AZ, NM, TX)." He concludes that there is a "Race Chasm" of states with black populations of 6 to 17 percent where Obama does especially poorly.

There are several problems with the graph. The first is that the X-axis is not black population, but the rank of black population among the 33 states Sirota considers. While that choice makes the graph look nicer, it potentially obscures what's actually going on in the data. Here's a replication of the Sirota graph with black population on the X-axis:

As you can see, the results look somewhat less clear. Now let's deal with the second problem -- the fact that Sirota treats caucus and primary states identically. As readers pointed out, the U-shaped pattern of Obama support against black population, which I noted in my first post on the subject, goes away when you exclude caucuses, which are (a) public (potentially limiting the influence of race on voting) and (b) favorable territory for Obama given the organizing strength of his campaign. Here's a replication of Sirota's graph with caucus states excluded and black population again on the X-axis:

All of a sudden the racial chasm starts to look a lot less cavernous. An even more useful approach is to specifically consider white support for Obama as measured by exit polls (the real dependent variable of interest) rather than looking at aggregate results. When we do this for the same set of states Sirota considers and fit a linear trend to the data, there's no evidence of a "race chasm" -- sadly, there seems to be instead a negative linear relationship between Obama's white support and black population (correlation= -.47):

When you add back in New Hampshire and the heavily Hispanic states and drop Florida, which I think is more appropriate, the relationship is even stronger (correlation= -.53):

In short, the claim that Obama does uniquely poorly with whites in states with moderate black populations is not well supported.

Update 4/2 10:24 AM: Sirota pointed out a typo in the data for Florida that is now fixed in the graphs above.

Update 4/3 9:53 AM: Josh Marshall links to this post, writing that "I'm not sure this apparent disagreement is anything other than the same point expressed a different way":

[R]acially polarized voting increases with the size of the black population in a given state. That leaves Obama winning a lot of states with few blacks. But once the black population gets into the high single digits, racialized voting kicks in and Obama then can't get enough of the white population to win. Only when blacks approach 20% of the population does the black population get large enough to make up for and often overcome the increased white resistance to voting for Obama.

I'd tend to agree with that story -- apologies to Sirota if I misinterpreted his point.

I should also clarify that, as a commenter points out below, Washington DC is not included in the last two graphs. The reason is that there is no exit poll data available that I've seen on the breakdown of the white vote. If those numbers are available, please send them to me.

Update 4/3 1:06 PM: One way to address Marshall's comment is to consider the rate at which Obama gains black votes and loses white votes as states have more African Americans. This plot graphs Obama's total white and black vote separately as a percentage of the white+black primary electorate:

Obviously, this won't work as well in states with significant Latino and Asian populations, but it's one relatively simple way to look at the data.

Also, a commenter below is concerned that I'm wrongly imposing linearity on the data, so here is an identical plot to the one above with a flexible polynomial instead of a linear fit -- there's some evidence of the relationship flattening out but I'm not sure how much I believe it. There's certainly no "chasm" effect in white support:

Finally, a commenter wondered about the relationship with the South excluded -- here's a plot that also excludes all of the states that were part of the Confederacy (one traditional definition of the South):

Update 4/3 5:59 PM: Here are two more graphs that may be of interest. The first graphs Obama's total white and black vote separately as a percentage of the white+black primary electorate again but this time with flexible polynomial fits instead of linear ones:

The second tries to look at combined white and black support for Obama across the range of black populations by state. See what you think:

(Note: There are so few African Americans in the extremely white states that the exit polls can't estimate the rate at which they voted for Obama. So I put in 85 percent as an approximation to avoid having those states be missing from the graph.)

Update 4/3 8:02 PM: Marshall discusses Sirota's column and my post in a TPM video.

Comments

[Seth]

Great post, Brendan.

[Mark]

In fact, it seems to me that your analysis supports Sirota's. You show that Obama does better among white voters in states with a smaller black population. Since he also tends to do very good among black voters there are two types of states Obama wins: Those with overwhelmingly white population because there he can draw more on white support; and those with large black population because there he doesn't need many white votes to win. He loses the states in which there are too few blacks to carry him alone but too many for Obama to gain much support by white voters. This is what the original graph tells us and it seems consistent with at least your last two graphs.

[Jinchi]

I was also skeptical when I saw Sirota's graph, but particularly because 4 of the 10 states he listed (RI, CT, MA, NJ) were in Clinton's backyard and a 5th (FL) was essentially uncontested.

[KCinDC]

Why is DC excluded from the last two graphs (and not even noted)?

[Matt Weiner]

sadly, there seems to be instead a negative linear relationship between Obama's white support and black population

How much of this is driven by the south? The lower right corner of the graph is occupied by southern states with high black populations, and it wouldn't be surprising if the racial dynamics in the south were different from those in the north. (This isn't just stereotyping, it's well supported by for instance the Poole and Rosenthal work.) I'm curious if there's still a linear relationship between Obama's white support and black population among non-southern states. (Or among southern states, for that matter.)

Since southern states tend to have more black people, if southern white people are less likely to vote for Obama that could create the appearance of a linear relationship even if the driving factor is southernness rather than black population.

[Matt Weiner]

Actually, eyeballing it it looks like the trend line among Northern states could be a really sharp downward slope from about midway between NH and NM to NJ, with DE and MD as extreme outliers -- and they're arguably border southern anyway. Hm.

[Eric E]

I'm pretty concerned about using a linear trend to show that there's no chasm - a line can't possibly support that kind of shape, so essentially your linear fit is just smoothing through that chasm.

On the other hand, political campaigns and election results are notoriously sensitive to all kinds of factors (read, time-dependent and parameter-dependent in a highly non-linear way), so a model like Sirota's that attempts to model a particular behavior (support for Obama as a non-monotonic function of the African American proportion of the total population) is probably a bigger assertion than this uncontrolled dataset can really support. In that case, your correlation model is probably more appropriate in that it merely asserts some relationship between African American proportion of the total population rather than trying to model a particular functional relationship.

In short, I think we're probably really seeing the result that support for Obama is positively correlated with African American proportion of state population, but all we can really firmly conclude is that black folks are more likely to support Obama than white folks, and I don't think the data can really be said to firmly support the "chasm" conclusion.

[jake]

interpretating

=

interpreting?

[Matt]

Unless you used a statistical method that isn't otherwise noted, it is inappropriate to attach a linear trend to your data. Linear regression assumes that the observations are independent from one another. However, with this data, we can be fairly certain that the observations are strongly related to one another (e.g. after every victory, Obama received more material support which increased his viability in the next contest).

[Brendan Nyhan]

Matt -- If there is serial correlation in the data due to Obama doing better over time, it could make the standard errors too small in a linear regression, but I'm not interpreting confidence intervals here.

[Ryan]

This may be impossible to discuss without more information, but what does your analysis portend for the general election? Let's assume for a second that the demographic trends we've seen in the primary are replicated in the general election (a dicey proposition, I know). All of a sudden, the black share of the electorate would be halved in many states -- if blacks made up about 50% of the Dem primary vote in South Carolina, for example, then they would make up about 25% of the electorate in the general, right? Would this spell huge trouble for Obama, because he would lose moderately black states *as well as* heavily black states in the general? I thought the Wright controversy was a joke but I'm quite surprised and saddened by the underlying trends we've seen hold up throughout the campaign. I want to make sure I'm not misreading the data.

[Matt Weiner]

Ryan, all I know about this is what I read on blogs, but my guess is that whether white Democrats vote for Clinton against Obama doesn't necessarily predict whether they'd vote for McCain against Obama. Obama will probably lose MS, LA, GA, and SC in the general as white Republicans turn out; but he'll probably win MA, RI, and (I predict) NJ in the general as the Democrats who voted for Clinton return to Obama in the general.

And in the general, the blacker the state, the more that the white people vote Republican -- even if the nominee is white. See Tom Schaller, who says that Bush's margins were greater in blacker states because of party-line voting by southern white people. Schaller casts cold water on the idea that Obama could put any of those southern states into play; but by the same token, there's no reason to think that Rhode Island (which gave Bush 23% approval before it was cool) will vote Republican no matter who the Dems nominate. (I've seen people make noises about polling putting Massachusetts in play, but I don't believe it -- you can find polls that put Texas in play for Obama too.)

[Realist]

Is it possible to include black support for Obama in your analysis? As I said in the TPM thread, black support for Obama in MA, RI, FL, TN also appears to be significantly below average. Wouldn't this imply causes other than racial polarization for low white Obama support in those states?

[jake]

Interpretating? Shouldn't that interpreting?

[molasku]

It's not clear why one would expect a "chasm" only across white support rates -- certainly not one implying an increase in white support in the South. Sirota's "chasm" is in overall support because he means the gap among winning support coalitions, where the overall margin is key. A fairer framing of your graphs is that they support Sirota's points that weakness in white support is tied to black population and that the weakest white support in the South is offset by the larger black vote.

[Juris]

Awesome post.

I have obsessed with this matter since reading about it on Brad DeLongs site. I'm glad to see someone else realize that the Caucus data is a different animal and should be considered an outlier for this analysis. Thanks for your rigorous work.

I think that your green and orange graph is perhaps the most important graph you presented, because it's easier to see what I think the data is really trying to say.

But I do have a small quibble about that graph. Why did you include the Caucus numbers in this one? Without the caucus data it would make even more sense. My recommendation would be to overlay -- on the Green/Orange graph -- the two regression lines from your chart at the Update at 1:06 (Obama White and Obama Black votes). This would, more emphatically, show that the white vote goes down less than the black vote goes up, as a function of "percent-black-population."

Thanks for the most rational and rigorous discussion of this data I've seen -- it is such a relief to see that critical thinking is still alive. I salute you.

By the way, you got a great write-up at Daily Howler from Somerby today.

[frankly0]

I'm not sure I've never seen a more crackpot analysis of polling data, with the exception of Sirota's original stupidities.

Look, do you not understand how much you are trying to pull out trends from an absurdly small amount of data, especially when you restrict yourself to primaries? Throw out the always idiosyncratic and far left wing VT, then you really only have UT and WI to try to show any kind of trend at all -- and WI neighbors Obama's home state of Illinois, and how many Democrats are there altogether in UT - ten?

And what happens when you take into account the fact that all the cases showing lowest white support for Obama occur in the South, with an electorate racialized for well known historical reasons?

And what happens when WV, with only 3% black population, goes big time for Clinton, as we all know it will?

Some "analysis".

[Juris]

franklyO --

I think you're being much too harsh on Brendan.

My take on Brendan's analysis was that he was, in effect, criticizing Sirota's analysis which was laughable --- or would have been if it weren't so prominently discussed on Marshall's TPM site as well as others. I think Brendan appreciates (as much as you or I do) that there is not a lot of data and the scatter is pretty large. I think he was doing all of us a huge service by diving into the data and proving to us that there is NO "Race Chasm" as Sirota was claiming.

[frankly0]

Juris,

My point is simple. Given the limitations of this data set, it's essentially impossible to draw any useful inference at all -- even agreeing with Josh Marshall's point, which Nyhan quotes, is groundless, because there is so very little data to support the claim that states with very small numbers of blacks tend to have large white support for Obama -- as I said, there really are only two or three states that might support that claim, and each one has its own weakness as an example. And, of course, even that "correlation" will be destroyed when WV kicks in.

People should know when they don't know something. And I don't see how this data leads to any particularly interesting inference at all -- hardly surprising when there are so few data points to begin with.

[Juris]

frankl0 --

I agree with you when you write:

"And I don't see how this data leads to any particularly interesting inference at all.."

I still think that the importance of Brendan's post was that it debunks Sirota's "analysis" and lets people(a few?) that it's not as simple as Sirota and Marshall seem to think to parse this data and make sweeping generalizations about racism as the driving force of Clinton's campaign -- nor is it rational to do so.

As far as I know, Brendan is the only one who has taken the time to rationally criticize the pseudo-analysis of Sirota. I think he deserves our appreciation for taking this on.

[Jinchi]

This analysis also misses an equally important factor in the Democratic race - that Hillary Clinton is the first woman with a real shot at the presidency. Clinton's numbers have been particularly weighted by the votes of women.

In Massachusetts, for example, she won women 62-36, Obama won men 49-48. Similarly, In Georgia she won 62% of white women, but only 49% of white men. In most races where I've seen a breakdown, Hillary does 10-20% better among women than men of all ethnic groups (except for African Americans). Brendan's charts (and Sirota's) would be quite different if they accounted for that additional variable.

Then you've got natural regional differences as well. My guess is if you did this analysis for previous elections, where neither candidate is black or female, you'd still get a significant skew in the numbers.

[Kristen]

I find it amazing that race is considered a bigger factor among Southern whites, than religion. The South is the most religious part of the country. Protestant churches in the South are overwhelmingly Republican ... the national black church, Southern or not, is overwhelmingly Democratic.

What you are measuring is white non-religiousity, not race.

[John Egan]

I am concerned by an error introduced by BOTH x-axes.

Rather than % black population either in actual percentage or rank order - - I am curious about African Americans among a specific state's registered Democrats - since it is that percentage that historically informs the internals of the state party and many public perceptions of that party's dynamics - racially informed or not.

Even though many white independents may have voted in various state primaries, they did so within the context of perceived racialized party politics - especially with Southern Dems. The Mississippi Democratic Party is a case in point.

I would be curious to see how these graphs and regression lines would look if they were adjusted for percentage of African Americans among registered Dems. I suspect, however, that getting reliable figures is quite difficult.

[ikl]

I feel like a broken record, but this whole analysis is incredibly simple-minded. White people aren't some monolithic voting block that is similar across all states - the white populations in different states differ a lot from one another. Southerness looks like a pretty good predictor of whites voting for Clinton; southerness + low education levels is an even better predictor. Western states and greater New England (minus New York state and working class cities in Southern New England) tend to be Obama friendly. Appalachia is very strongly for Clinton. So are Hispanic areas in the Southwest. Presence or absense of black folks doesn't seem to make a difference to white voting behavior within any of these regions, as far as I can tell.

As was pointed out above, when WV goes strongly for Clinton (that is, if she is still in the race then), that is going to be a big outlier - but one that could have been predicted beforehand by anyone who has been paying attention. Kentucky as well. Which suggests that % of blacks in the state is not really what is driving white voting behavior . . .

[dr anonymous]

People should know when they don't know something. And I don't see how this data leads to any particularly interesting inference at all -- hardly surprising when there are so few data points to begin with.

Franklyo, I think the point is that many of us who have been following this primary know that something interesting is going on--whether it's Sirota's story or not--and are trying to explain it. If the data doesn't explain it, that doesn't mean that the story is wrong--it just means that you can't make a compelling case for the story with the data. Which is fine...data is overused anyway.

[Jinchi]

Which is fine...data is overused anyway.

That's a strange take on this whole discussion. Franklo's point is that the statistics are being oversimplified to prove a point that many people believe in their gut, absent any evidence at all. It wasn't too many months ago, that Brendan cited arguments proving that Obama had a "ceiling" in white support of about 35%.

That idea was very popular at the time among people who were certain that white voters - deep down - simply weren't ready to pull the lever for a black man. (It's also part of Hillary's electability argument against him). It has now been completely repudiated by Obama's successes across the country.

In that case, statistics were misused to support the argument - data wasn't the problem, abuse of the data was.

Statistics can be a powerful tool, but when you find your results confirming your inherent biases, it's doubly important that you confirm that your data actually mean what you think they mean.

http://www.brendan-nyhan.com/blog/2008/01/an-obama-ceilin.html

[p_lukasiak]

There are lots of different ways to look at the data -- Sirota's was one of them, and while not perfect was "valid" as far an any kind of analysis goes.

The problem wasn't Sirota's graph -- it was his claim of a causal relationship between what he claimed was the "racist" Clinton campaign, and what the graph showed.

The fact is that in South Carolina, where the whole "Clinton/racial campaigning" think was all that the media focussed on for a couple of weeks, Obama did somewhat/signficantly worse among black voters than in subseqent states where blacks made half (or reasonably close to half) of the primary electorate. The worst he did in those states was (IIRC) 85%.

In fact, the whole phony race accusation stopped Obama dead in his tracks (using SUSA polling data here)-- after lagging well behind Clinton in white votes, he finally started to white votes in SC while clinton lost them -- right after the Iowa caucuses (polled Jan 4-6), he was at 29% to Clinton's 38%...when two weeks prior to that, Obama had only had 18% of the white vote.

The next SUSA poll (taken Jan 16-18..and the tail end of the controversy) showed that Obama had lost 7 points among whites, and Clinton jumped to 50% among whites.

Meanwhile, although Obama had started strong among black voters, and climbed steadily, the Jan 4-6 showed an additional 12 point jump in black support for him (to 69%) -- without the whole "racial controversy" in sight. The Jan 16-17 poll showed Clinton losing only 3 points among black voters (to 20%) and Obama picking up 4 points. (to 73%).

Its pretty obvious that Obama's black support in SC was based on positive racial identification (I mean, Edwards with only 2-3% of the black vote? after taking 37% of the black vote in 2004, and making economic justice and poverty central themes of his campaign?), and that he was making strong inroads in the white vote before the controvery. Once the controversy died down, Clinton lost all the white support she had gained (and then some).... but almost all of it wound up with Edwards. From the time the controversy was over, to the election, Obama only gained back two points in the white vote -- windin up at 24%.

IMHO, black people realized that the racial campaigning charge was BS, and ignored it -- it didn't hurt Obama in terms of black support, but it didn't help him either.

But it killed him with white voters, who knew the charges were bogus as well, and saw exactly who was pushing them (the Obama campaign and its supporters) and weren't about to let him get away with it.

[Tonymil]

One major flaw in all this analysis is that it assumes race is the only factor in this primary when in fact gender has been a huge factor. These charts completely fail to take gender into account and are terribly skewed as a result. The assumption is that Obama has been losing the white vote in states where the black population is greater than 6 percent. But, if you separate white female voters from white male voters, the assumption falls apart. Obama has lost the white female vote in almost every state, so that is a constant. But Obama has won the white male vote in many states, including states like California, Connecticut, Georgia, Maryland, Massachusetts and Virginia, states with black populations greater than 6 percent. So these charts don't give a fair picture of what has happened in these democratic primaries. And they are an even worse predictor for a general election between Obama and McCain. In fact, the white vote may flip-flop in the general election. White women, given a choice between Obama and McCain, may overwhelmingly choose Obama, and white men may overwhelmingly choose McCain. So I don't see the utility of these charts in either analyzing the democratic primaries or predicting results in a general election.