A Quick Poll Round-up, And It's Ugly

Obama Job Approval

We recently were treated to a rash of polls on Obama's job approval ratings. As I discussed earlier, these reported poll results are getting less and less reliable. This latest round of polls really underscores my point nicely:

49% Approve,  47% Disapprove   =   +2%      Gallup  (1500 adults, 11/14 - 11/16)

44% Approve,  54% Disapprove   =   -10%     Rasmussen (1500 "likely voters", 11/12 - 11/16)
43% Approve,  50% Disapprove   =   -7%       Reuters / Ipsos (1586 adults, 11/7 - 11/11)
45% Approve,  47% Disapprove   =   -2%       CBS News / NY Times (1495 adults, 11/6 - 11/10)
45% Approve,  48% Disapprove   =   -3%       Economist / YouGov (2000 adults, 11/6 - 11/10)

The numbers are all over the place, despite all being held within 10 days of each other. This is clearly outside the reported 3% margin of error for these polls (the margin of error in this case means that 95% of the time the spread is within X% of the true underlying distribution in the population). In fact, if you take these polls at face value, this distribution implies a margin of error closer to 9% - hardly the kind of number that inspires confidence. However, let’s look at an earlier set of polls from each pollster.

Previous Polls

49% Approve,  48% Disapprove   =   +1%      Gallup  (1500 adults, 10/24 - 10/26)

46% Approve,  52% Disapprove   =   -6%       Rasmussen (1500 "likely voters", 10/18 - 10/20)

43% Approve,  51% Disapprove   =   -8%       Reuters/Ipsos (1586 adults, 10/15 - 10/21)

45% Approve,  46% Disapprove   =   -1%       CBS News/NY Times (1495 adults, 10/4 - 10/8)

44% Approve,  48% Disapprove   =   -4%       Economist/YouGov (2000 adults, 10/9 - 10/13)

Observed Change

+1%  Gallup

 -4%  Rasmussen

+1%  Reuters/Ipsos

 -1%  CBS News / NY Time

+1%  Economist / YouGov

This group is also widely scattered, still closer to the 9% margin of error than the reported 3%. However, there's a very good reason for that - the amount of change within each pollster is much smaller, with a likely margin of error between polls of about 4%. So what we are seeing is more likely a distribution of different bias from different pollsters. Now, this bias is unlikely to be intentional or necessarily reflect the views of the pollster. It is much more likely that the pollster is putting together some magical weighting process that is their own secret sauce to weight the poll respondents. They want to get it right, they just don't know how. So which pollster is right? Well, you, lucky reader, just might be able to find out. I am working on solving that problem right now, though it might take me a little while to be sure. Like maybe three months, when the first primary results come in to provide a little ground truth.

Democratic Primary

Let's be complete, and give the same treatment to the Democratic primary, and see where the race sits. 

52% Clinton,   33% Sanders,   5% O'Malley    CBS News / NY Times (11/6 - 11/10)

56% Clinton,   31% Sanders,   2% O'Malley    FOX News (11/1 - 11/3)

57% Clinton,   35% Sanders,   4% O'Malley    McClatchy / Marist (10/30 - 11/5)

53% Clinton,   35% Sanders,   0% O'Malley    Quinnipiac (10/29 - 11/2)

The quick summary of these polls show Clinton up by 21% over Sanders, with a possible margin of error around 6%. The interesting thing is that this is actually much closer to the reported margin of error of 4.5% (since the sample sizes are much smaller than the job approval ratings, the reported margin of error goes up). Could it be that the pollsters are better at modeling registered Democrats than the population as a whole? Possibly. Or this group of pollsters just happen to agree with each other more, especially when they have one less dimension to weight their respondents with (party affiliation). 

Now look back at the previous results for these pollsters.

Previous Polls

46% Clinton,   27% Sanders,   0% O'Malley    CBS News / NY Times (10/4-10/8)

45% Clinton,   25% Sanders,   1% O'Malley    FOX News (10/10 - 10/12)

      (data not available)                                     McClatchy / Marist

43% Clinton,   25% Sanders,   0% O'Malley    Quinnipiac (9/17 - 9/21)

Observed Change

  0% CBS News / NY Times

+5% Fox News

  0% Quinnipiac

Not much reported change in the spread. However, in all of these polls (that we have data for) both Clinton and Sanders seem to have increased their portion significantly (by 6-10%!). This could reflect a firming up of the electorate, with a shrinking pool of undecided primary voters. With only about 10% of the voters still in an 'undecided' camp, there really isn't much room left for Sanders to get a majority. If he's in this to win, he needs to start changing some minds. The one area that voters don't really know about Sanders is his foreign policy. That also happens to be a subject thrust into the forefront in recent days. Expect Sanders to start heavily detailing his stance on Syria and ISIS. Anything less, and he's just playing for the Veep slot.

Republican Primary

OK, this is just a mess. The margin of error is a statistic specifically calculated for a two way race. When you have someone like O'Malley on the Democrats' side who is in the low single digits, it really isn't much of a stretch to make the old formulas fit "well enough." However, the Republicans have four major players right now, with another six making up about 17-25% of the voters. There's no way that any self-respecting mathematician could continue to use "margin of error" calculations, yet the polls keep rolling out these meaningless numbers, none-the-less. 

First, the "raw" polls:

26% Trump   23% Carson  11% Rubio  11% Cruz   4% Bush   17% Other  Fox News (11/1-11/3)

23% Trump   24% Carson  12% Rubio    8% Cruz   8% Bush   20% Other  McClatchy/Marist (10/20-11/4)

24% Trump   23% Carson  14% Rubio  13% Cruz   4% Bush   13% Other  Quinnipiac (10/29-11/2)

23% Trump   29% Carson  11% Rubio  10% Cruz   8% Bush   14% Other  NBC / WSJ (10/25-10/29)

28% Trump   23% Carson  11% Rubio    6% Cruz   6% Bush   11% Other  IBD / TIPP (10/24-10/29)

Treating this as a horse race between Trump and Carson, they appear essentially tied, with Trump reportedly getting less than 1/2% advantage. Taken at face value, Rubio would seem to have a 2% lead over Cruz, while still polling 12% behind Trump or Carson, basically garnering half the support of the front runners. Even without an easy sense of the "noise" on these polls, it's easy to see clear stratification of the race, at least according to the pollsters' models.

Previous Polls

24% Trump  23% Carson    9% Rubio 10% Cruz   8% Bush  17% Other  Fox News (10/10 - 10/12)

     (data not available)                                                                          McClatchy/Marist

25% Trump  17% Carson    9% Rubio   7% Cruz 10% Bush  20% Other  Quinnipiac (9/17 - 9/21)

25% Trump  22% Carson  13% Rubio   9% Cruz   8% Bush  16% Other  NBC / WSJ (10/15 - 10/18)

17% Trump  24% Carson  11% Rubio   6% Cruz   8% Bush  21% Other  IBD / TIPP (9/26 - 10/1)

Observed Change

  +2 Trump     0 Carson   +2 Rubio   +1 Cruz   -4 Bush     0 Other   Fox News (10/10 - 10/12)

   -1 Trump   +6 Carson   +5 Rubio   +6 Cruz   -6 Bush     0 Other   Quinnipiac (9/17 - 9/21)

   -2 Trump   +7 Carson    -2 Rubio   +1 Cruz    0 Bush    -2 Other   NBC / WSJ (10/15 - 10/18)

+11 Trump    -1 Carson     0 Rubio     0 Cruz   -2 Bush  -10 Other   IBD / TIPP (9/26 - 10/1)

Looking at the earlier polls from the same firms, it's clear that the noise on the Republican primary is much larger. Whether that marks a true volatility in the race, is an artifact of sampling the sheer number of candidates, or represents a real instability in the pollsters' models, these polls require a healthy dose of skepticism when reading the "latest breaking poll" on the Republican race.

But what is the latest breaking poll? And how should we interpret it?? Well... I'll keep you posted. 

Late Breaking News!

As of the time of this post, Bobby Jindal has just suspended his campaign. Given that most pollsters actually didn't find a single voter supporting his candidacy, this will have absolutely no effect on the race whatsoever.