Friday, January 22, 2016

Drunken Polls

The Subtle Joys of Polling Crosstabs
Most pundits are happy to report the top line from a poll, blithely reading off that a candidate is ahead by 20% one day and behind by 15% the next. If there is any internal concern or irony in these reporters, it's long gone by the time the final version is posted. Only on particularly slow news days will you ever hear some tidbits about "second choice options" or other simple cross tabs. However, at Black Swan, it's those buried analytics that are most fascinating to us, even if we realize that sane people consider the inner workings about as interesting as webpage source code.

Fortunately for you, we have found an intriguing set of cross tabs that I think you're going to love! (If you are still reading at this point, thank you for your faith in me, or just your morbid curiosity.) Public Policy Polling (PPP) has surveys that go deep into the mind of likely voter to find their first and second choice for the Republican nomination, as well as several match-ups in two-way or three-way races. The results are fascinating, and not from a strictly political sense.

Is that Jim Gilmore Supporter Drunk, or Just Stupid?
No, this is not a slight against Jim Gilmore, or anyone who thinks that he would make a good president. This is about one specific person in New Hampshire who was contacted by PPP. I can talk about just one individual, because there is only one person out of 515 Republicans who supports Gilmore (okay, maybe that was a slight against him).

This one supporter, we will call them Gil in honor of the candidate, is easy to track through the poll, and see their various answers. For instance, Gil identifies as a moderate Republican, who is male and over 65. Of note is that when asked to name a second choice, Gil prefers Ted Cruz. However, in a 4-way race between Cruz, Rubio, Trump, and Bush, Gil changes his mind to Rubio. Then again, if you remove Bush from the contest, to a Cruz-Rubio-Trump contest, he's back to Cruz again. Switch the 3-way to Cruz-Rubio-Bush, and Gil likes Rubio again. Hey Gil, you do know that you can still vote for Cruz, right? Alright, clearly this guy likes Rubio and Cruz, but which one does he like better in a head-to-head match up? Ted Cruz. So if the race were just Cruz vs. Trump, you would vote for.... Trump. Of course you would, Gil. Hey, can you tell me how many fingers I'm holding up?

This goes on and on, with this one loon contradicting himself every chance he gets. It turns out that when asked if he has a favorable or unfavorable opinion of each candidate, he doesn't even like Jim Gilmore. So clearly this guy is unreliable, and probably just messing with the pollster. You're bound to have a few outliers who are adding just a little extra to your margin of error.

Let's switch to Iowa, where things are bound to be better, right? Things are looking good for the Gilmore campaign there, as PPP found TWO Gilmore supporters out of the 510 likely Republican caucus-goers surveyed. On another positive note, only one of them has an unfavorable view of Big Jim (the other supporter still isn't so sure). Also, both of these supporters belong to a critical group of voters that the Republicans would like to win over in the general election: liberal female tea-party members. Jim Gilmore does seem to attract the most interesting people.

Cruz Supporters Only Slighter More Sober
It's a good thing that there aren't many of those crazy people responding to the pollsters. For instance, Ted Cruz is a serious candidate with serious popularity, and you can count on his supporters not to be messing around like Jim Gilmore's. At the very least, 100% of Cruz's supporters in both states view him favorably. However, it does kind of beg the question of why it is that, just two minutes later, when the pollster asks New Hampshire about the four-way match up between Bush-Cruz-Rubio-Trump, only 83% are loyal enough to Cruz to pick him a second time, while 13% defect to Trump. That's roughly seven different people out of the 55 or so Cruz supporters contacted. It's ok, though, because in a two-way race between Trump and Cruz, most of them jump back into Cruz's camp. They do seem to lack a certain courage of conviction.

Rubio doesn't have it any better. For any given match-up involving Rubio as an option to vote for, anywhere from 12-15% of his supporters in New Hampshire choose another option. (The one exception is a head-to-head against Trump. Apparently there are some things you just don't joke around about.) In Iowa, that number for Rubio is closer to 17-20%. In another camp, Bush can hold on to about 90% of his supporters through any match-up in NH, and between 82-88% in IA. Trump is apparently the only candidate who has a strong cadre of no-nonsense supporters, maintaining 95-98% no matter how many alternatives they are given.

America is Sick
Across the board, at least 10-15% of the voters surveyed give weird and contradictory answers over the course of a single survey. Keep in mind that these are the likely voters, according to the experts' models. So it's not like they would lie, or not know who they were voting for, right? Instead, I have a few hypotheses about what is happening:
a) These respondents are severely inebriated in some way
b) Contrary to earlier reports, Iowa is not a native English speaking state
c) There is rash of under-reported concussions sweeping New Hampshire
d) Bored respondents are just handing the phone to their two-year-old child/grandchild to play with
e) The most jaded political pundits are right, and the American electorate literally does have the memory of a goldfish

No matter what the cause is, it should be clear that there is an additional source of error that goes far beyond the mathematical sampling error that is being routinely reported and used to justify these results. Furthermore, this previously unreported "bat-guano-crazy error" is somehow making it through the "likely voter" screens (which apparently don't include questions like "Have you been drinking this evening?"). So the next time that you see a headline about a candidate jumping 20 points in the polls, consider what the people they polled might have been smoking.

(For those interested in seeing more of the insanity, you can look at the New Hampshire and Iowa polls directly.)


1 comment:

  1. Based on your previous posts, does this mean that you can quantify the indecisive ratio for each candidate and apply that to analysis of polls along with the variables that you have outlined in the past?

    ReplyDelete