As polls evolve, can they maintain credibility?

Nate Silver, the guru of all polling, has some insights.

There is no shortage of reasons to worry about the state of the polling industry. Response rates to political polls are dismal. Even polls that make every effort to contact a representative sample of voters now get no more than 10 percent to complete their surveys — down from about 35 percent in the 1990s.

And there are fewer high-quality polls than there used to be. The cost to commission one can run well into five figures, and it has increased as response rates have declined.¹ Under budgetary pressure, many news organizations have understandably preferred to trim their polling budgets rather than lay off newsroom staff.

Cheaper polling alternatives exist, but they come with plenty of problems. “Robopolls,” which use automated scripts rather than live interviewers, often get response rates in the low to mid-single digits. Most are also prohibited by law from calling cell phones, which means huge numbers of people are excluded from their surveys.

If you click on any one link up there, make it the footnotes listing (the tiny '1' that offsets the rest of the line). Silver also explains how demographic weighting is used to improve the model to compensate for fewer poll respondents, and how it can also damage the credibility of the pollster.

How can a poll come close to the outcome when so few people respond to it? One way is through extremely heavy demographic weighting. Some of these polls are more like polling-flavored statistical models than true surveys of public opinion. But when the assumptions in the model are wrong, the results can turn bad in a hurry. (To take one example, the automated polling firm Rasmussen Reports got fairly good results from 2004 through 2008, but has been extremely inaccurate since.) Furthermore, demographic weighting is an insufficient remedy for the failure to include cellphone-only voters, who differ from landline respondents in ways that go beyond easily identified demographic categories.

Texas suffers from all of these developments because, as one of the most extreme non-voting states as well as an expensive set of media markets that scares away all but the most wealthy and/or most craven and corrupt, we are left with poor ballot options and the associated 'inexorable' meme.  And that's when we aren't getting random results, like the fellow with the most common name winning the primary.

Just no way to run a democracy, is it?

I and others have spent many millions of pixels decrying the development of online polling, but the truth is that it's all we have left.  But it's also not all that bad, either.

Internet-based polling has been a comparative bright spot. In fact, the average online poll was more accurate than the average telephone poll in the 2012 presidential election. However, there is not yet a consensus in the industry about best practices for online polls. Some online methods do not use probability sampling, traditionally the bedrock of polling theory and practice. This has worked well enough in some cases but not so well in others.

But all of this must be weighed against a stubborn fact: We have seen no widespread decline in the accuracy of election polls, at least not yet. Despite their challenges, the polls have reflected the outcome of recent presidential, Senate and gubernatorial general elections reasonably well. If anything, the accuracy of election polls has continued to improve.

My excerpts aren't doing justice to Silver's full piece here.  Go read it. He continues where I left off with some very meaty analysis of why polling of primary elections gets it wrong so often compared to general elections.  Good stuff.  I'll skip to the end, picking back up on the explanation of demo-weight.

Demographic weighting is a legitimate and necessary practice. The past decade or so has seen stronger and stronger partisanship, stronger and stronger alignment of voting in different states, stronger correlations between up- and down-ballot voting (there are fewer split tickets than there used to be), and stronger predictability of voting behavior on the basis of demographics. All of that makes demographic weighting more powerful. It has become easier to project election outcomes on the basis of informed priors — without conducting polls.

If my hypothesis is right — the relatively steady accuracy of the polls is the result of the increasing demographic predictability of elections helping to offset lower response rates — we could see a disastrous year for the polls if and when political coalitions are realigned. A black or Hispanic Republican presidential candidate could scramble the demographic coalitions that prevailed between 2000 and 2012, as might a moderate blue-collar Democratic nominee, or a certain type of third-party candidate. None of these things is especially likely to happen in the near term, but the current political coalitions won’t hold forever. The 2012 presidential map looks fairly similar to the one in 2008, or 2004, or 2000, for instance, but rather different from the one in 1996 or in years before that, when states now seen as locks for one party or the other were considered swing states instead.

Disapproval of Congress results in higher voter turnout, according to Gallup... who whiffed badly in their 2012 presidential results.  That might be something interesting to note in late October, as we start to get early-voting numbers from the Harris County Clerk's office (and elsewhere).

Here's where I'm going to plug the onliners in which I participate: YouGov -- which is the outfit that the Texas Tribune/University of Texas use -- and which is polling right now for governor, US Senate, and Congressman.  Head on over and sign up to participate.  And Politix, which is national but woeful in its obvious conservative bias.  (Nobody actually cites them as something approaching reputable, FWIW.  They don't even call what they do a poll, but a debate.)

Real Clear Politics does something similar to Silver, which is aggregate and average several polls to produce a "poll of polls", as CNN refers to it.  Note in the Texas group of polling conducted so far in the race between Greg Abbott and Wendy Davis, Rasmussen -- with its bright red flavoring -- showed Davis closing the gap between them to 8 points.  This suggests that the contest is even closer than the most recent poll reveals.  Nobody in the media has dared to say that, though.

We go into the homestretch of the 2014 midterm elections with the polling we have, not the polls we wish we had or might have at some future time, as Donald Rumsfeld said (paraphrasing).  They're still better than paying attention to what the Talking Heads on teevee say.

But if we should be skeptical of the polls, we should also be rooting for them to succeed. One of the reasons news organizations bother to conduct expensive surveys is to serve as a check on the misrepresentative opinions of elites, including those of their own reporters. Even a deeply flawed poll may be a truer reflection of public opinion than the “vibrations” felt by a columnist situated in Georgetown or Manhattan.

Click on that last link there.  And LYAO.