New poll results look much better for Kerry:
Democratic presidential nominee John Kerry gained ground at President George W. Bush's expense in polls taken in Florida, New Jersey, Iowa, New Mexico and New Hampshire after last week's presidential debate.
Kerry has a 2 percentage point edge in Florida and the candidates are tied in New Hampshire, two states that were among those Bush won in 2000, according to the American Research Group.
In New Mexico and New Jersey, Kerry leads by 3 percentage points, within the error margin, according to polls. Bush and Kerry are tied in Iowa. The three states backed the Democratic nominee in 2000.
“Independent voters, who shifted to Bush from Kerry beginning just prior to the Republican convention, seem to be shifting back to Kerry, and that trend has intensified in the days following the first debate,'' American Research Group President Dick Bennett said in an e-mail.
One's tempted to say this is the start of major Kerry “momentum.” Trouble is, I have come to mistrust all the polls. In order to believe one, I'd need to know lots more, starting with what percentage of the electorate they think will vote, and how that number compares to 2000. I think that turnout, at least in the 'swing' states, will be substantially greater than four years ago, especially among the younger voters. Is this reflected in their models?
Then again, I suspect that exit polls got the Florida vote right (showing a Gore win), and the actual count got it wrong — in substantial part due to the “butterfly ballot” people who were polled but not properly counted. But with exit polls we don't have any doubt about who the 'likely voters' might be, so the sampling problem is easier.
i think that “likely” voters in most polls are suspect for the reason you mentioned. Especially polls such as gallup, which defines likely as those who voted in the last election…htmmm…what about the 18-21 year olds?
From what I can tell based on comparing past election polls to actual results, zogby is the closest, but it only reflects the last two elections, which thus could be just attributed to luck.
More statistically anecdotal evidence (anecdotally statistical?) here:
http://online.wsj.com/public/resources/documents/info-battleground04.html
To me, the only relevant statistic, at least at the outset, is the number of polled individuals…its surprisingly often I see one of these majorly touted national polls with less than 1000 people surveyed…this is statistically insignificant in a nation of more than a quarter billion people…why not flip a coin?
I always thought it was Groucho, someone told me its Aaron Levenstein, but that only makes it more relevant:
Statistics are like a bikini – what they reveal is suggestive, but what they conceal is vital.
No? Too 50’s? Too yankee erudite? Then how about the Florida version:
Clem asks Abner, “Ain’t statistics wonderful?” “How so?” says Abner. “Well, according to statistics, there’s 42 million alligator eggs laid every year. Of those only about half get hatched. Of those that hatch, three-fourths of them get eaten by predators in the first 36 days. And of the rest, only 5 percent get to be a year old because of one thing or another. Ain’t statistics wonderful?” Abner asks, “What’s so wonderful about statistics?” “Why, if it wasn’t for statistics, we’d be up to our asses in baby alligators!”
Christopher Chopin wrote: “To me, the only relevant statistic, at least at the outset, is the number of polled individuals…its surprisingly often I see one of these majorly touted national polls with less than 1000 people surveyed…this is statistically insignificant in a nation of more than a quarter billion people…why not flip a coin?”
The size of the sample is less consequential than the question of whether your sample is an accurate cross-section of the population you are attempting to make inferences about. That’s really Michael’s question–the polls, in order to be accurate, have to represent the people who will actually be voting on November 2. But you don’t need a sample of 2 million in order to have an accurate poll. In fact, I refer you to the embarrassing example of the Literary Digest poll in the 1936 election where they telephoned 2 million people and predicted that Landon would clobber FDR. We know what happened, FDR won, but I’ll test your lawyerly abilities here and ask you to guess why the huge sample didn’t matter?
First, Literary Digest didn’t poll 2 million, they polled 10 million, but only to their mostly republican readers, and got 2 million responses. Next, let’s keep in mind the Literary Digest was dead on as to the presidential winner in 20, 24, 28, and 32 with the same polling system. Then lets recall that in 36, while Landon didn’t get the predicted 54% of the popular vote, he still got nearly 40%, and yet only 8 electoral votes to FDR’s 523.
Gallup on the other hand, who made himself famous in ’36, polled 50,000 random people. And yes, he predicted that FDR would win, by less than 10% of the votes.
Did any of this help predict an electoral college ignoring the results of the popular election almost completely? Was gallup’s prediction as to Landon’s 44.3% of the popular vote helpful in ’36? Not by a longshot. So at best, you can say that in ’36 gallup predicted the winner and L.D. didn’t, but gallup had by their own records over a 6% error, and was only differing in 8% of the votes from the L.D. poll itself.
So it seems to me what you’re saying is its almost as good to poll 10 million republicans as any 50,000 random people for the sake of polling accuracy. Ok, doesn’t give me great confidence, but maybe. Now how does that reflect on the current Gallup poll’s use of about 800 voters? 621 in one previous one I saw? And if it doesn’t, then why don’t we just ask any 2 people?
Or any one?
Or just flip a coin?
Christopher,
From your post, I think we agree that sample size alone is of secondary consequence and can move on from there. I’m not suggesting at all that it is worthwhile to make inferences from a clearly biased sample. You do so at your own risk. How far off the Gallup poll was from the LD poll depends on (1) the level of sampling bias in the LD poll due to the poorly conceived sampling strategy (by the way, LD did change its sampling strategy in ’36–going from mail cards to telephone), and (2) the degree of random sampling error in the Gallup poll. Because of sampling error, scientific polling in no way guarantees that its results will be perfectly accurate. Only that human error or bias is not responsible for any disparities between what’s observed and how things really are. The LD poll is a case study for why we should not be impressed by large numbers alone. It is crucial to get an accurate cross-section of the people who will actually go to the polls, and the best (but not perfect) way is through a random sample of people drawn from those who will vote.
I’m not trying to convince you to blindly trust the polls, but neither should you casually dismiss scientific polling. Used properly (i.e., with care and discernment), it can offer good insights.
I don’t see how you can dismiss the method of the science as secondary to “scientific polling.” The question asked and the size and variance of the group polled are all three equally relevant.
Now admittedly, statisticians would have you believe that percentage of error should fluctuate equally for any sample of any group larger than the sample, requiring, for example, 1067 tests for a + or – of 3% margin of error on any population of any size, or, in reverse, the margin of error = 1 divided by the square root of the number of people in the sample.
But even under this somewhat unbelievable analysis, 1,600 people sampled will give you a 2.5% margin of error, not a 3%, as with 1,067, or a 4%, as with the Gallup poll I referenced. You’re talking about an 8 point swing. When even the Literary Digest Poll was only 8 points from lining up with Gallup’s in ’36. So how can an organization that makes this much of its bread from polling not bother to narrow the margin of error? If 621 people were enough of a sample in ’36, why did George Gallup sample 50,000 before guaranteeing a winner?
And in a race that’s dead even, what scientific use, at all, is a poll with an 8 point swing?
You raise a valid point–in a race that is [presumably] very close, scientific polling is not going to discern fine distinctions without a larger sample to boost statistical power.
You are correct to believe that polling services should do everything they can to limit error. I’m not disputing that. But the improvement in accuracy diminishes as the sample grows larger. For instance, you get much more improvement going from 100 people to 1000, than from 1000 to 5000. But the cost per subject remains the same, so I don’t fault polls for trying to be cost-effective. They have to pay their bills, same as we.
But that doesn’t really answer your belief that a scientific poll at this point is of no value, because the race is so close. But I actually don’t know that the race is so close, because the various polls have odd procedures for screening respondents. For instance, they exclude those who only own cell phones as opposed to land lines. Some commentators, e.g., at Center for American Progress, have described the arbitrary nature of the screening criteria for determining likely voters. I trust scientific polling if I believe the sample is an accurate cross-section of whatever group the pollsters are trying to study (i.e., people who really will vote Nov. 2) and if the questions are well designed. I will even agree that if one is looking at a nearly dead heat, a larger sample will be helpful. But back to the Gallup poll, reducing the margin of error for a biased sample will still yield a garbage result. This isn’t the fault of scientific polling or statisticians, but the judgment of the pollsters.
It looks to me as if polling of ‘likely voters’, however determined, is just not going to be an effective predictor this year. Both sides are doing an unprecedented amount of registering and GOTV of people who have never previously registered or voted.
The ministers of every Pentacostal Holiness and (white) Baptist church in my county are preaching from the pulpit that Christians must vote for Bush — something that hasn’t happened here in such a systematic and brazen way in previous elections. The result is bound to be a significant bloc of first-time voters. On the other hand, every day last week at Dem HQ people in their seventies came in to fill out registration applications for the first time in their lives, all of them saying “We’ve got to get rid of this guy.” Registration of 18-21-year-olds locally is at levels not seen in the last 20 years. With Virginia’s non-partisan registration (no way to show a party preference even if you want to…grrrr), we have no indication of which way these new young voters will go. Very few of them can be reached through land lines.
These realities make me glad I’m not in the polling business for a living. And they help keep my mind off mood-swing-inducing polls and on doing the things we can do that win elections: identifying supporters, activating them, and getting them to the polls. Identifying undecideds, making an effort (mailing, visits) to persuade them, and following up.
Nell,
My wife and I were looking at different churches after moving to New York and ran into the same thing! The politicization of religion ought to be a productive thread indeed, should anyone ever think of doing it.
PS to Michael–sorry for the irrelevant post.
I am not interested in politics. Neither Kerry or Bush wins is okay.
But who can give you happines and benefit, just yourself. Don’t doubt about it.
“GOD,” if that is your real name:
As a student of Islamic mysticism (aka Sufism) I should like to familiarize you with one of the fundamental metaphysical principles of the great Sufi master Muhyi ‘l-din Ibn al-`Arabi’s cosmology: There is no repitition in [God’s] self-disclosure [i.e. the cosmos] (la tekrar fi ‘l-tajalli)
Ergo: You are not God.
Is it too late to change my vote on blocking nonsense?
WHY W. DOESN’T GO TO CHURCH.
Empty Pew
by Amy Sullivan
Check it out at: http://www.tnr.com/doc.mhtml?pt=vKO1twmWG2Uvnyi2qoWQfW%3D%3D
Gallup is now down to polling 515 people for a 5 point + or – and a 10 point swing…with 2 points difference between Kerry and Bush…talk about C’ing YA.