Posted by: Dark Defender | October 24, 2008

Tina Fey (and the MSM) dont understand what polls are

Last week on SNL Tina Fey (playing Sarah Palin) mock disregarded the polls saying (from my memory not a direct quote) “I don’t pay attention to polls after all they are just a systematic method of predicting election results”

The line meant to mock the supposed idiocy of Governor Palin but instead what it provides is a glimpse of how the unquestioning devotion that the MSM and Washington elites put in polling. 

This is apparent if you follow the political news.  Every article for the past two weeks is based on a poll, Democrats lauding them, Republicans explaining their flaws (as yes I am here), MSM types explaining what “The American People” (a stupid phrase which means nothing when you think about it) will do on election day based on them.  Its nauseating, lazy and does a tremendous disservice to the country.  This is both a result of and the cause of the proliferation of polling which MIchael Crowley wrote an excellent article in The New Republic, I highly recommend you read.

What the MSM misses however is a basic point, polls are not and never have been intended to be a prediction of election results, they are and only can be a snap shot of voter opinion at that time.


If polls were designed to predict election results, how can we explain this?

If polls were designed to predict election results, how can we explain this?

Further polls are not magic.  They are made by people, and the results they spit out are only as good as the methodology used in creating the poll. 

The first, most important and most difficult part of creating a poll is selecting a sample, you not only need to find a group of people statistically relevant from a sufficiently diverse in geography, race, ethnicity, gender, economic class etc to reflect the country (or state as a whole) but you also need to then be able to predict which of those people are going to actually vote. 

When you recall that only roughly 2/3rds of the population votes (in a high turnout year) and half of those contracted by pollsters refuse to cooperate, it should be clear this is an extremely difficult task.

Jay Cost at RCP has an excellent article on why the polls are so variable this year, you should read it!

A relevent quote:

So, we have made three observations: (a) relative to 2004, the standard deviation for Obama and McCain’s polls are high, indicating more disagreement among pollsters at a similar point in this cycle; (b) the shape of the distribution of each candidate’s poll position is not what we might expect; (c) multiple polls are separated from the RCP average by statistically significant differences.

Combined, these considerations suggest that this variation cannot be chalked up to typical statistical “noise.” Instead, it is more likely that pollsters are disagreeing with each other in their sampling methodologies. In other words, different pollsters have different “visions” of what the electorate will look like on November 4th, and these visions are affecting their results.

Think of it this way. Suppose there is a bag of 130 million red and blue marbles that all the pollsters are sampling from. One pollster will pull a sample of 750 marbles, another a sample of 2,500, and so on. Oftentimes, they are going to pull different results from the bag. One pollster might pull 53% blue, another might pull 52%, and so on. However, as long as they are all pulling marbles from the same bag, the results will probably not differ too wildly. And after enough time, the distribution of those pulls should look something like those idealized pictures of Candidate A.

However, what if each pollster had a slightly different bag s/he was pulling from? In that situation, we should find more divergent results. That’s basically what I’m suggesting here – that the bags the pollsters are pulling from are different. That’s producing some of these larger-than-expected variations.

Now, I want to be clear: I am not making any claims about which pollster has the better sample of the electorate. I’m not singling anybody out for being right or wrong because frankly I do not know. I’m just pointing out that there seems to be disagreements among them that cannot be explained by random variation.

Importantly, there is one thing that the polls do not disagree on, the fact that Obama has a lead. All the polls show that. Also, we might begin to see convergence here soon. If pollsters have different methods for predicting what the electorate will look like, those methods might produce similar-looking “electorates” by the time we get to Election Day. At least for now, though, there is disagreement – not about who has the lead, but about how big that lead is.

This I think gets down to what this election is going to be all about, and well what every election is about. 

Who votes, and who do they vote for?

The Obama people claim they have registered millions of new voters and will be getting massive turnout from college kids and minorities.

The McCain people are trying to turnout the evangelicals who shocked the MSM by coming out of nowhere and turning out in record numbers for Bush in 2004.

Whoever is right is going to win this election.  Its that simple. 

Polls however cannot tell us this.  The pollsters have no way to know who these people are and even if they are found, half of respondents don’t cooperate with pollsters and even when they do what they tell pollsters and what they actually are not necessarily the same. 

Also keep in mind even if someone has come up with a way to perfectly sample these people, we have no way to know who that is now, and may not even after the election.

This is an election which is going to be determined by who turns out their voters, polls arent going to tell us that and the MSM should stop being lazy and start doing real reporting on the state of the race. 

With that in mind I am going to give a poll free analysis of the race, probably tomorrow.



  1. […] of the race and final election prediction OK as promised, I will now do a poll free analysis of the race, and as an added bonus I will give my oh […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s


%d bloggers like this: