A recent episode reinforces my distrust of self-styled “family” and “Christian” political groups:
The American Family Association set up a “poll” last month on their web site titled “America’s Poll on Homosexual Marriage.” The “poll” asked people whether they approved or disapproved of gay marriage. The AFA promised that “Results of this poll will be presented to Congress” — presumably as some kind of reflection of nationwide attitudes. (Of course, any attempt to suggest that the results of a self-selected “poll” were representative of the population would have been statistically unsupportable. While a properly conducted, random poll of sufficient size can be a valid statistical representation of the opinions of the population, a self-selected poll cannot).
The AFA eventually decided not to present the results to Congress, after gay marriage supporters learned of it and began voting in large numbers. The reason given for the change was that the “poll” now “represent[ed] something other than what we wanted it to.” (See the discussion at Volokh.com). However, if the “poll” had come back as strongly opposing gay marriage, we can assume it would have been presented to Congress as originally planned. What does this say about the AFA? One reporter suggests that “No such poll can be said to represent an accurate picture of popular opinion. But, clearly, the AFA had hoped Congress would take the numbers it planned to produce as exactly that kind of evidence.”
We can’t know for sure that the AFA would have used the “poll” as legitimate evidence of nationwide attitudes (can we?), but that conclusion seems quite reasonable, especially when we look at the title of the poll (“America’s Poll”) and the stated intent to present results to Congress. And, if this was the AFA’s intent (to present results as if they came from a legitimate poll), then the project is clear evidence of either incompetence or dishonesty. Either the AFA did not understand the first thing about polling, or they were planning on playing a rigged game — a clearly selection-biased “poll” — and then framing the results as having some kind of legitimacy.
I find it hard to believe that they just made a mistake (“We didn’t realize that’s not a statistically significant poll”). And so my assessment is that the AFA was planning to be dishonest, and the only reason they eventually chose not to was because they did not have the results they were after. That’s the typical dirty political fighting which we expect from many politcal groups. I know that many political groups play dirty (or even outright lie) all the time. But I find it disturbing that “Christian” groups would be so eager to discard honesty if it became a political liability.
Everyone knows these type of polls are “rigged”. Generally, you can vote as many times as you want and of course there is selection bias. Just compare the online polls by Foxnews and CNN. They are are almost always different because the channels have different viewerships. There are other examples of Christian groups playing dirty political games, so if this were just another example it would be of no surprise. I have to say, however, that my experience over the past couple of years in helping to get Ohio’s Defense of Marriage Act passed with these types of groups is that they are not always all that sophisticated. You have good, honest decent people who don’t understand politics, polling, etc. so it truly could be an honest misunderstanding of how things work.
When I worked on Capitol Hill we got these sorts of things all the time. They had roughly zero influence. If there is one kind of social science that political operatives tend to be VERY sophisticated about it is polling… ;->
Nate is exactly right about this. Any intelligent person in politics knows the difference between self-selected internet polls and scientific opinion polls. Around the year 2000 Time did an internet poll on the person of the millenium and Rick Flair won in a landslide.
In general polls done by people associated with politicians are even better than scientific polls done by the media. Any congressman who wants to know how his or her public feels about gay marriage can easily find out.
this particular poll was absolutely rigged. i voted in it (supporting gay marriage) when the poll first came out. a few weeks later i received an email (i had given my email address when i voted in the poll) informing me that the poll was tipping in favor of gay marriage and that i should email the poll to all my friends to make sure they voted too. clearly, the people behind this poll were seeking to achieve results that justified and mandated their particular agenda.