5. Find out how the sample was selected. If the subset chosen is linked to the quantity under study, the results will be biased.
6. The quoted errors are usually calculated assuming an independent sample selection, the true errors may be bigger.
7. How many people refused to take part? Could the balance of results be different among them?
8. Ask what post-survey adjustments were made. Professionals sometimes knowingly take selection-biased samples and then try to correct for them mathematically.
9. Did everyone tell the truth? Some things people are embarrassed to admit. Did everyone understand the question?
10. People will often give the answer they think the surveyor wants or expects. Are the questions worded to hide that? Are any of the questions loaded? Many issues are in shades of grey – do the tick-box categories cover all cases?
I could go on.
Incidentally, your point number 1 might not be true if the errors are positively correlated.
And on point number 2, scientific facts are not established by *any* sort of opinion poll, (unless you’re studying opinions).
I would also day that in respect of point no 2, the example given is historically inaccurate. Very few people in Galileo’s time believed the earth was flat.
The importance of polls on global warming is not to show which side is “right”, and which side is “wrong”; in that regard they are dispositive of nothing. Polls tell you whether your rhetoric is working or, as in the case of global warming alarmism, no so much.
What they tell you is that mockery of your opponents, instead of answering their objections, eventually devolves into mockery of the majority. You can say that you are tired of answering objections, then people conclude that you must think the issues in not nearly as important as you claim.
If you are losing in the polls, the message is not do more of the same only louder.
P.S. It seems to me that many of the posters here are more interested in feeling like a member of a “smarter” minority than creating a political majority that might result in concrete accomplishment.
“It seems to me that many of the posters here are more interested in feeling like a member of a “smarter” minority than creating a political majority that might result in concrete accomplishment.” -moptop
If being part of the majority didn’t require a science lobotomy i’d be all for it…
What we need is a science-y philosopher king state in which the science educated elite rule over the ignorant masses…
national physics care plan to include pre-emptive as well as emergency and in class round the clock coverage of all critical thinking needs…
January 24th, 2010 at 3:52 am
Thank you thank you thank you. I support this message wholeheartedly. This is a major pet peeve of mine.
January 24th, 2010 at 5:48 am
Not bad. But there are lots more.
5. Find out how the sample was selected. If the subset chosen is linked to the quantity under study, the results will be biased.
6. The quoted errors are usually calculated assuming an independent sample selection, the true errors may be bigger.
7. How many people refused to take part? Could the balance of results be different among them?
8. Ask what post-survey adjustments were made. Professionals sometimes knowingly take selection-biased samples and then try to correct for them mathematically.
9. Did everyone tell the truth? Some things people are embarrassed to admit. Did everyone understand the question?
10. People will often give the answer they think the surveyor wants or expects. Are the questions worded to hide that? Are any of the questions loaded? Many issues are in shades of grey – do the tick-box categories cover all cases?
I could go on.
Incidentally, your point number 1 might not be true if the errors are positively correlated.
And on point number 2, scientific facts are not established by *any* sort of opinion poll, (unless you’re studying opinions).
January 25th, 2010 at 3:16 am
Panel no 2 is historically inaccurate.
January 25th, 2010 at 3:21 am
I would also day that in respect of point no 2, the example given is historically inaccurate. Very few people in Galileo’s time believed the earth was flat.
http://www.bede.org.uk/flatearth.htm
January 25th, 2010 at 6:33 am
The importance of polls on global warming is not to show which side is “right”, and which side is “wrong”; in that regard they are dispositive of nothing. Polls tell you whether your rhetoric is working or, as in the case of global warming alarmism, no so much.
What they tell you is that mockery of your opponents, instead of answering their objections, eventually devolves into mockery of the majority. You can say that you are tired of answering objections, then people conclude that you must think the issues in not nearly as important as you claim.
If you are losing in the polls, the message is not do more of the same only louder.
January 25th, 2010 at 6:36 am
P.S. It seems to me that many of the posters here are more interested in feeling like a member of a “smarter” minority than creating a political majority that might result in concrete accomplishment.
January 25th, 2010 at 6:50 am
The necessity of this plea is obvious,
as is its vanity.
Georg
January 26th, 2010 at 2:17 pm
“It seems to me that many of the posters here are more interested in feeling like a member of a “smarter” minority than creating a political majority that might result in concrete accomplishment.” -moptop
If being part of the majority didn’t require a science lobotomy i’d be all for it…
What we need is a science-y philosopher king state in which the science educated elite rule over the ignorant masses…
national physics care plan to include pre-emptive as well as emergency and in class round the clock coverage of all critical thinking needs…
January 27th, 2010 at 7:23 pm
Is the lady news anchor wearing a ‘stache? How European!
February 15th, 2010 at 12:28 pm
[...] said, I’m skeptical the results of this so-called survey reflect [...]