Snatching defeat from the jaws of the MDC

By Phil Plait | May 26, 2009 8:04 am

As you may recall, the JREF recently tested a woman named Patricia Putt who claimed she could "read" people, that is, write down statements that accurately described these people, without knowing them in advance. She applied for the Million Dollar Paranormal Challenge, and the preliminary test was performed in England by JREF friends Professors Christopher French and Richard Wiseman.

Briefly, 10 women were read by Ms. Putt, she wrote down descriptions of them, and then after the readings each of the ten women was allowed to look over the readings and determine which one fit her best.

In advance of the testing, Putt and the JREF agreed that if 5 of the 10 women chose correctly, then this would indicate that something interesting was happening, and she could move on to the final testing. And how did she fare?

Not one of the women picked the reading that matched her. Putt scored 0 out of ten.

When she discovered this, she was shaken, and seemed fair about it. However, that’s now changed. On his blog, Richard Wiseman described what happened, and Putt has responded. As with almost all applicants who fail, she is finding ways to rationalize her failure. However, she goes much further than this, claiming that in fact she got 10 out of 10 right! How?

Because Ms. Putt made ten readings, and according to her each one of the women read did in fact choose a reading that she thought fit her best! In other words, Putt says that because each women did pick one of the readings, there must have been something in that reading that the woman felt fit her, and therefore Putt scored a perfect 10/10.

Um. Well, not so much. First off, if I write down 30 random character traits, of course everyone will find a few that fit them! This is plainly obvious, so having the women pick one reading that fit them best is no indication at all that Putt has any psychic abilities.

Worse, though, is that in the rules, each woman was required to pick one reading! So literally, Putt could have written "You eat puppies, you push little old ladies in front of cars, you pick your teeth in public, and you belch loudly in elevators," for each reading, and the women would have had to pick that. So this indicates nothing at all.

I also find it fascinating that in the comment, she says,

I should also like to point out that neither am I a winger or a whiner, so when I decided to take up the Randi Challenge I did so with both eyes open knowing that the protocols would be completely one sided in favour of JREF, and so it was.

I willingly walked into the lion’s Den knowing it would be a long, difficult and very tiring day with apparently nothing to smile about at the finish.

That’s interesting indeed, since she had to sign a form indicating that she agreed to the terms of the Challenge. As a later comment states, she had to sign a form that in part says, "I, the undersigned, agree to all terms and conditions listed in this document outlining the protocol for my preliminary test in the James Randi Educational Foundation’s One Million Dollar Challenge. I agree that the protocol outline describes a fair test of my claimed ability."

Seems to me that I wouldn’t sign such a form if I thought the test was unfair. And we know that many, many Challenge applicants try very hard to come up with reasons why they failed the test after the fact, despite the JREF jumping through many, many hoops to make sure that before the test the applicant is happy with it. No test is conducted unless the person applying is happy with it. That’s a basic and inviolable rule of the Challenge.

But what I find most interesting of all is that after the test, Putt emailed Alison Smith, who is in charge of the Challenge protocols, saying (as quoted by French in the Guardian (see below)):

With them [the volunteers] being bound from head to foot like black mummies, they themselves felt tied so were not really free to link with Spirit making my work a great deal more difficult.

Hmmm, which is it? Did she score a perfect 10 out of 10, as she claimed in her response on Professor Wiseman’s blog, or was it impossible to do the reading because the subjects were not free to be read?

Or is there a third possibility? I’ll leave that to your imagination.

It should be noted, as Professer French points out in his Guardian article about this, that "For the record, no volunteers were “bound” and Mrs Putt did not speak to any of the volunteers after the test. One can only assume that she picked up on their feelings of being “tied” via her psychic powers."

So make of this what you will. It doesn’t prove psychic powers don’t exist, nor does it prove that Putt doesn’t possess them if they do. But it does show that when you take out the bias, take out the feedback the reader gets from the client, and take out the aspects of the readings that allow someone being read to be led to the conclusions they want to hear, Putt scored zero.

Occam’s razor slices close indeed. You may try to bandage the cut after the fact, but you’re only covering up the reality of the wound.

CATEGORIZED UNDER: Antiscience

Comments (102)

  1. Interesting. You’d expect that she’d get a couple right by random chance alone. Obviously, she didn’t really want to win and used her vast psychic powers to make sure that she didn’t get any right. OMG, she’s psychic!

  2. It would seem to me that if she were trying to show that the protocols were flawed, a better approach would have been for her to say that the women deliberately rejected descriptions that applied in order to spoil her chances of winning. Even if the women had written down characteristics that applied to them ahead of time and put them in a sealed envelope, the possibility would exist that those descriptions would also be inaccurate.

    I think designing a fair and foolproof protocol is probably one of the biggest hurdles in any one of these tests. What the person being tested agrees to probably tells a lot about their critical thinking and reasoning skills.

  3. It seems to me that this is not the best medium!

  4. jh

    Reading the defenders of this woman on the Guardian article leaves me just confused.

    People seem to want to believe in woowooism so much that they think that mere REQUESTS for extraordinary evidence is a bad trait.

    It’s like they have this false dichotomy. Either you believe in fairy crystal bullcrap, or you are a cold human being whose rationality disallows any real fun in life.

  5. dhtroy

    Wait. I’m picking up a psychic connection from the other side … they’re saying … she’s … full … of … s…

    Dang, I lost the connection.

    I really need a new Spiritual Carrier, AT&T Spectral services is horrible.

    :P

  6. @jh

    Either you believe in fairy crystal bullcrap, or you are a cold human being whose rationality disallows any real fun in life.

    Hush up, put on this dull grey robe and get back into the dull grey factory.

  7. craig sachs

    I allways enjoy the excuses more than the test. I remeber seeing Randi on thats incredible when I was a kid and hearing the excuses of a telekenetic page turner and pencil roller.

  8. Mchl

    A suggestion: why not publish Ms.Putt’s readings in a form of ‘Pick one that describes you best’ poll? I wonder what would be the distribution of results?

  9. Daniel J. Andrews

    I watched (the Amazing) Randi debunk dowsers (video was on youtube). They were happy with the test, thought it was fair, and when asked how they thought they did, they were all confident they’d done well in detecting water in the pipes and a mineral in boxes.

    Then Randi showed them they hadn’t scored much better than 10% (random guessing), and asked them how many thought they had an ability to dowse. All of them put up their hands despite the evidence of their failure right there. The video didn’t specify why the dowsers thought they still had this ability. That would have been interesting to hear their justifications.

    What I’d also like to know is there anyone who has taken the MDC and failed AND then admitted they weren’t psychic after all? Or do they all try to justify their failures? This aspect of people fascinates me.

  10. sleepneed

    @dhtroy

    Get Verizon Lifeless, I only have a few life zones in my experience.

  11. Hi Phil

    It’s not good enough to get someone to sign that they agree with the test before they start, you need to film them saying it openly and honestly to the testers.
    This way you can show, not only that they signed it and agreed with it, but how they were feeling (confident, calm, happy, whatever) when they agreed. This would show more than a signature.

    Just my thoughts.
    Nathan

  12. Mrs. Putt claims she can detect personality traits by hearing the voice of a person, she explains it using spirits and paranormal stuff, but the claim seems quite natural for me.

    I don’t undrestand why is that cold reading is either a psychic ability or a con. We’ve lived millions of years among other humans and being able to detect subtle cues from the way they speak, or by fishing the right answer has obsvious evolutive advantages..

    Cold reading is like a sleight of hand trick or juggling: it can be used for bad, but it could have a great entertainment value if we could, instead of dismissing/believing it as paranormal, we could promote it as extraordinarily human.

  13. @Daniel J. Andrews

    after you spent years believing yourselv to be special, built a reputation on grows used to defend it daily, it’s hard to change your own mind.

  14. Sespetoxri

    As though she would have any other response- either she’d privately think exactly this, or publicly say it. There was no way possible she would accept she was simply wrong. Woo is it’s own self-fulfilling prophecy- wrong thinking supports itself in all things, before and after it is dis-proven.

  15. TheBlackCat

    @ alexandre: That is not what she claimed. She claimed that she can establish a psychic link with someone through hearing that person’s voice and use that to not only pick up on personality traits, but determine specific details about their life and past. It is not just a matter of cold reading.

  16. @alexandre van de sande

    The reason that cold reading is typically either viewed as “psychic” or a “con” is because people generally underestimate the abilities of humans to do a number of different things. That explains, at least, the “psychic” part of your question. As to the “con”, it’s viewed as a con solely when people claim that they can do these things by contacting the dead. Thus far, no evidence has been presented that they are, in fact, contacting the dead. Since the same can be accomplished through ordinary means, then it is most likely a con.

    On the other hand, there are a number of entertainers who use cold reading in their stage acts, purely as entertainment. Take for example the Banachek or the JREF’s own James Randi (when he actively performed). Still others use cold reading as part of their everyday profession. Salespeople would do well to understand the principles and techniques of cold reading to boost their sales.

    We merely dismiss it when the person using the ability claims that it is paranormal.

  17. Tom K.

    Just another small medium at large.

  18. Samsam

    Perhaps I am simply a nasty person, but…

    Don’t we all use our strengths to our advantage? Wouldn’t anyone who honestly had a psychic power use it, subtly, secretly, to enrich themselves? This wouldn’t need to be evil, though it easily could be. Wouldn’t public knowledge of the special ability somewhat neutralize its effectiveness (at bettering our lot in life).

    If you want to find psychic powers, look at rich/powerful people who keep a low profile. Anyone who makes a big deal of it is likely a fraud.

    Note that I don’t believe any such abilities exist.

    Samsam

  19. Kees

    I’d already read about these results. Inspired by them I recently conducted the following little experiment on a family camping trip: My niece was reading her horoscope and I told her (basically) that it was bull. To strengthen my point me and a few other cousins played a game of ‘guess your horoscope'; Someone reads out the horoscopes in a random order and the other participants have to guess which horoscope for the day belongs to them. Obviously this game is only played when the day is drawing to a close.
    The results were… predictable

  20. Patricia Putz is more like it!
    Feh!

  21. Larry

    I could have predicted that this is the way this matter was going to turn out before it even began and I don’t even claim to be psychic!

  22. I was once involved in a dialogue with an astrologer. When a test similar to the one Ms. Putt underwent was proposed, the astrologer replied that those kinds of tests never work because most people will subconciously avoid choosing the most accurate reading. Kind of a, “the truth hurts,” position. Ms. Putt has obviously much to learn about the subtle art of obfuscation and avoidance of reason. How much you wanna bet she never goes through with her promise to retake the test in a year’s time.

  23. IVAN3MAN

    Like in this cartoon, some people only hear what they want to hear…

    Some people only hear what they want to hear.

  24. Ha! And here I thought I’d be able to hold up Ms. Putt as an example of both failing gracefully, and accepting the consequences of modern scientific rigor in the face of personal feelings.

    The fact that JREF requires a signed statement — declaring that the supposed psychic finds the test protocol to be fair/unbiased — is brilliant. A month or two later, she goes back on the statement? The dissonance is glaring! Clearly she’s either lying or suffers a cognitive impairment.

    Now if only we could do something when the dissonance occurs in those who hold high office…

  25. Ema Nymton

    Holy crap, Occam, that link is an amazing source of stupidity!

  26. jh

    @Ema

    Yeah. I read it, and typed up a good page and a half of response to it.. then just let it go.

    People don’t seem to understand statistics very well around there.

  27. Yeah, when she first started showing up on the JREF forums, I just couldn’t help but to shake my head (and I admit I had some unkind thoughts regarding her sanity and connection to reality).

  28. Greg in Austin

    @Occam,

    Can you summarize that article, or at least point to a link of the article that is not printed in white characters on a black screen. After looking at it for 2 minutes, I think I’ve gone partially blind. Thanks!

    From what little I could read, it seems some people complain that the tests are not fair to those who claim to have paranormal abilities of course. But how can the claimants call foul if they are involved in, and agree to, the method of testing?

    8)

  29. Greg in Austin

    Samsam said,

    “If you want to find psychic powers, look at rich/powerful people who keep a low profile. Anyone who makes a big deal of it is likely a fraud.”

    Not good enough. If you do any research on a rich person, and see any signs of hard work, good education, determination, financial skills, financial inheritance, social promotion or just a matter of being in the right place at the right time, etc., those (and more) would all be more likely than psychic powers. Otherwise, you could say that the person’s success is really attributed to the invisible pink unicorn living in the garden.

    8)

  30. Mchl

    @Greg,

    When having problems reading pages like this one, just copy whole text to your favourite word processor

  31. Greg in Austin

    @Mchl,

    Thanks for the tip. I’m quite capable of doing that, or setting my web browser options to force all web pages to a particular font, font size, font color, background color and whether or not to load background images. I could probably write my own java program to read the entire website, convert every letter to its binary coded decimal equivalent, and output the sound of each One and Zero as a pulse thru the computer’s speaker.

    However, in this case, not only have I gone blind from the inverted contrast, I think my head exploded from actually reading the content! I haven’t looked it up, but I’m certain there’s a proportional relationship between the readability of a website and the level of bunk contained therein.

    8)

  32. MadScientist

    Anyone ever notice these sorts of patterns:

    I’m not against vaccines but …

    I’m not a whiner but …

    I really am psychic but …

    Oh that’s right – it’s known to fishing folk as “the one that got away”.

  33. Romeo: Actually, the odds of getting none right in that trial are about 35%. That’s not too shabby, really. The expectation value for this test is really just one right, so none right is hardly a surprising outlier.

    (Incidental tidbit: if you increased the number of subject to an arbitrarily large value, the probability of getting none right limits on 1/e ~ 37%.)

  34. Cairnos

    Tom K said “Just another small medium at large”

    Well a good medium is rare.

  35. Bill

    Well done, Cairnos.

    :)

  36. Greg the Goofy Antiscience Guy

    Patricia Putt’s nonsensical “rationalisation” (I use the term begrudgingly) of her failure aside…

    Phil wrote: “Seems to me that I wouldn’t sign such a form if I thought the test was unfair.”

    Herein lies one of the major problems (or advantages, to Randi and the JREF) of the MDC. Most people don’t sign the form, because it *is* “unfair” (at least in terms of being a true test of whether something interesting is going on). When they don’t apply, they’re said to be avoiding Randi.

    So when they’re irrational/deluded people who apply, they get debunked. When they’re intelligent and discerning (questions of genuine powers aside), and do what Phil suggested they do if they thought the test was unfair, they’re labeled as cowards/charlatans. You can’t have it both ways. And it hammers home the point that – in its current form – the MDC is meaningless. It’s a publicity tool, pure and simple.

    Phil: “In advance of the testing, Putt and the JREF agreed that if 5 of the 10 women chose correctly, then this would indicate that something interesting was happening”

    And there’s the crucial point. 1 out of 10 would be chance. Why was the level of “something interesting” set at 5 out of 10? I would think 4 out of 10 would be pretty interesting, and worth testing further. Given that this was a preliminary for the MDC, not the actual final challenge, why set the bar so high?

    The MDC is a sham. Snake oil sold by one of the best in the business.

    @Ema, jh and Greg in Austin: Excuse the stupidity of my website, I’m just a crazy woo intent on blowing out the candle of rationality. We all have dreams. Now, moving on from your initial responses, what exactly is incorrect or wrong about the MDC article?

  37. zar

    @Occam:

    Wow. That link was… wow. That’s a high concentration of crazy. I give it two Timecubes!

    @Greg:

    The problem with your story and methodology is that, rather than try to prove genuine psychic abilities in some sort of legitimate scientific way, paranormalists stomp their feet and complain that the skeptics are mean and that scientists are closed-minded for wanting evidence. Basing beliefs on evidence is why we no longer sacrifice people to the sun god or accuse others of witchcraft.

  38. Greg the Goofy Antiscience Guy

    @zar:

    “The problem with your story and methodology is that, rather than try to prove genuine psychic abilities in some sort of legitimate scientific way, paranormalists stomp their feet and complain that the skeptics are mean and that scientists are closed-minded for wanting evidence. Basing beliefs on evidence is why we no longer sacrifice people to the sun god or accuse others of witchcraft.”

    I’m discussing the MDC, and it’s lack of “some sort of legitimate scientific” validity. I’ve pointed out its flaws. Please respond to those points, rather than making vague and irrelevant comments about “paranormalists” stomping their feet, and the wonderful present in which we no longer sacrifice people.

  39. nankay

    The JREF doesn’t come up with the protocol all by itself, it is a give and take between the foundation and the person taking the test. If there is something either party doesn’t like, there is a counter offer made. It goes back and forth until both parties are happy.

  40. jh

    @Greg the Goofy Antiscience Guy Says:

    When they look for a number that is interesting, they go to what the testee says, and they come to an agreement.

    When she said that she could do it to anyone, she was in fact making a statement of 10/10. Not 5/10. 10/10.

    So they came down to 5/10.

    And even if it was 4/10, like you suggest.. it never even got that far.

    Personally, if you want to prove you are some psychic or similar, to me, you would have to defeat odds of over 1 in 500,000,000.

    Even the odds of winning the powerball are 1 in 195,000,000 or so.. and someone does that once or twice a month. To prove you’re a psychic, you have to defeat not only the odds of chance, but you have to blow them away.

  41. Greg the Goofy Antiscience Guy

    @jh: “Even the odds of winning the powerball are 1 in 195,000,000 or so.. and someone does that once or twice a month. To prove you’re a psychic, you have to defeat not only the odds of chance, but you have to blow them away.”

    Difference being 100 million people aren’t being tested for the MDC each week…y’know, while we’re discussing a lack of understanding of statistics and all. ;)

    But overall, I agree. To prove that there is such a thing as psychic ability, there needs to be some very strong scientific evidence. Exactly my point in fact, illustrating why the MDC is worthless.

  42. I don’t get it. Psychics can supposedly connect with the spirits of people who’ve died and been buried six feet under in a wooden box – or worse, cremated – but a black graduation gown is enough to block the spirits of the living??!!!! It’s a genuine mystery.

  43. @Greg the Goofy Antiscience Guy Says

    You spend a lot of time talking about statistical significance and outrageous odds but all that required of the claimant is to do what the claimant claims they can do under controlled circumstances.

  44. Joey Joe Joe

    It’s not particulary clear from her response, but is she claiming that each of the 10 women chose a different reading from the other? If that is the case (which I am almost certain it isn’t), and each woman was given the full 10 to choose from, then I’ll admit it is an interesting result.

    My probability theory isn’t quite up to scratch but I put the probability of that happening through random chance at:

    10! / (10^10) = 0.00036288

    Corrections appreciated if I got that wrong.

  45. I don’t even see why Ms. Putt is trying to rationalize the test. It only reduces her credibility. “Methinks Thou Dost Protest” and all that.

  46. It seems to me that the JREF goes through quite a bit of effort to come up with what certainly seems like fair tests. (I read “Flim Flam!” cover to cover and loved it.) If people had the “gifts” they claim they had, these tests should make it easy for them to prove it.

    What all these exercises in futility prove to me is that there are no “psychic powers” — even in the people who supposedly have them truly believe that they do.

  47. Comparing the Million Dollar Challenge to various experiments is ludicrous too. The Ganzfeld experiment looks for some statistical significance over hundreds or thousands of tests that the results may or may not be within the statistical deviation anyway. The MDC test an individual’s claim on what he or she says they can do.

  48. Rift

    >Excuse the stupidity of my website, I’m just a crazy woo intent on blowing out the candle of rationality.

    First intelligent thing you’ve stated yet.

  49. TheBlackCat

    @ Greg: I’ll take a shot at your article:

    Chances, of Anything…

    First, when dealing with very small sample sizes very large margins above pure chance are needed to guarantee there wasn’t a fluke. When you only have 10 samples, getting a few right by luck is not that exceptional.

    Second, the sorts of claimants the test deals with are not massive meta-analyses dealing with experiments at the very edge of statistical significance. The test deals with people who claim to have very high, if not perfect, reliability. People like John Edwards, who is confident enough to have his own show where he does this full-time, or people like Sylvia Browne, who directs tens of thousands of dollars in police resources and charges hundreds of dollars a session. Your meta-analysis example is completely irrelevant to the sorts of people actually being tested by JREF, because they claim to have reliability far above the threshold set by the test. They are being tested at a level far below the level they claim to be able to perform at. If sylvia browne had a success rate less than 10% above chance then she should not be directing police resources.

    Rule number 3 is also misrepresented. There are a lot of people who might take issue with their particular claim being labeled as “paranormal”, for instance those insanely expensive speaker wire salesmen he offered to test. Also, a lot of people have long-winded and irrelevant rationalizations for how their abilities might work. The point is that JREF is willing to test people having abilities they think are paranormal, and they really don’t care how the subject thinks it works. All that matters is that both sides agrees on what the final, visible result should be.

    Your second point is just more of the same.

    The second point is arguing against a standard liability waiver that is similar to one my school required when going on field trips, and giving giving James RanDi Educational Foundation the right to use the materials they gathered during the test for the purpose of the organization. Further, they need that kind of line in there, otherwise subjects could sue to suppress the evidence of the results then make up any story they want.

    The claim about not trusting Randi is also without merit. He does not participate in the tests unless requested to do so by the claimant. If you don’t like Randi, get someone else to do the test.

    That’s all I have gotten through so far, but it does not look promising.

  50. Rift

    @TheBlackCat

    You deserve kudos for getting that far…

  51. @TheBlackCat
    Yeah, I was a bit flummoxed by the criticism of the standard liability waiver. Sounds like scraping the bottom of the barrel for any perceived criticism by this point.

  52. M

    I thought I was a medium, but I turned out to be an extra large. Sorry, I could not resist.

    On a serious note, I was surprised that Ms. Putt agreed to take the test, and even more surprised when she initially seemed to be graceful in failing. It seemed odd that she would accept her total defeat so magnanimously. I guess I should have waited a bit longer before forming an opinion on the later.

  53. I know this was just the preliminary challenge, but to be thorough, shouldn’t there have been a control group of women who weren’t “read” by Putt who also were asked to choose the descriptions?

  54. Greg in Austin

    In post #38, Greg the Goofy Antiscience Guy Says:

    @Ema, jh and Greg in Austin: Excuse the stupidity of my website, I’m just a crazy woo intent on blowing out the candle of rationality. We all have dreams. Now, moving on from your initial responses, what exactly is incorrect or wrong about the MDC article?

    I did not say that website was “stupid.” I said it was hard for me to read. I used to program on a VAX workstation that was a green monochrome screen. But that was in 1992, not 2009. Eyestrain is a major concern nowadays, but luckily we have more than 4 billion colors to choose from.

    I also said that if one were to look at websites that promote anti-scientific viewpoints, especially information that has been debunked, one would see a common theme in colors and page layout style. For examples, take a look at Coast to Coast AM, Enterprise Mission, and Carpenoctem.

    As to the actual content of that article, as I said, I didn’t get 3 paragraphs into it before I quit reading. Occam said it was an excellent article, but did not elaborate, despite many requests to do so. I, and several others, asked a very obvious question which neither Occam nor Greg the Goofy has answered. So, I will ask again, and I will rephrase my question:

    If someone “claims” to have a supernatural ability, and goes to JREF with their claim, and goes thru the entire process of determining a reliable and repeatable method to test that claim, and signs a form that says they agree that the test method is fair, why do they then claim the test was unfair in some way AFTER the test is complete?

    8)

  55. @Greg:

    I’ll tread where others have tread already, but…

    The point of the MDC is not to establish that paranormal abilities exist (or don’t exist, for that matter) somewhere out in the world. The point is to challenge those who claim to have paranormal abilities to demonstrate it, in a way that is agreeable to both parties- at least initially. Of course the challenge is predicated on the belief that such abilities do not exist, but individual challenges are not meant to support such universal statements (which are not really testable).

    Think of it this way: I tell you that I can make a 300% return on monies invested with me. In order to test my claim, you can invest a little bit of money with me and see if I can make the claimed return. Now, even if I do get that return, it may just be chance, or I may be manipulating the results (for instance, for small sums of money, I merely pull out 3x the sum from a bank account). So, passing such a trial may not prove that I can always get a 300% return, but failing the trial is a pretty good indication that I can never get a 300% return. For the purposes of testing my claim, it is not relevant if I can get a return that simply beats the market average, because that was not what I claimed to be able to do. If I only claim to be able to beat the market by a small yet statistically significant amount, you may not find that such an impressive claim to bother letting me compete for a prize. Others can test the claim if they so like, then… but until someone does, my claim remains unsubstantiated.

    If a would-be challenger thought that given challenge criteria was unfair and the Randi organization was intractable, then it is incumbent upon that individual to explain what is unfair and what their idea of a fair test is. If the challenger wants a test in which the only important threshold is “statistical significance” and the MDC is not willing to offer her that test, then the challenger should at least announce that.

    Most won’t of course, because mere “statistically significant” ability does not rise to the level being interesting to the common person- much less worth paying for. Not many people would want to pay for someone who advertises their paranormal ability as just “better than chance”. I suspect many (as in the case of Ms. Putt here) don’t even have an idea of what constitutes statistical significance.

  56. Greg the Goofy Antiscience Guy

    @TheBlackCat: Thanks for taking the time to give a substantive response, appreciate it. Points below.

    “First, when dealing with very small sample sizes very large margins above pure chance are needed to guarantee there wasn’t a fluke. When you only have 10 samples, getting a few right by luck is not that exceptional.”

    Yes, I know that and agree wholeheartedly.It means that the smaller the sampling, the less accuracy you have in determining a claim (ie. to obtain statistical significance in a small sample, the bar is set extremely high to avoid ‘lucky hits’, whereas a much larger sample size allows for a more reasonable p-value to achieve statistical significance). That’s precisely my point about the MDC being worthless as any sort of scientific test. If she got 3, 4 or even 5 out of 10, Putt may have just been lucky. In getting 0, she may just have been having a bad day. The result proves nothing, except for Putt’s ability on that given day. I don’t see the point, it adds nothing to our knowledge base.

    “Second, the sorts of claimants the test deals with are not massive meta-analyses dealing with experiments at the very edge of statistical significance. The test deals with people who claim to have very high, if not perfect, reliability. People like John Edwards, who is confident enough to have his own show where he does this full-time, or people like Sylvia Browne, who directs tens of thousands of dollars in police resources and charges hundreds of dollars a session.”

    You final point is an excellent one (see my next reply for the intial point), and I said as much in my recent post (on my site) about the Putt challenge – if you’re earning a living from doing it, some might say that 5 out 10 is a bare necessity for the information to be useful (others would argue, such as the effect of so-called “dazzle shots”, although I’d still say less than 5 out 10 in this test would suggest people be very cautious in acting on anything a medium says). Although the MDC still proves nothing – even the most talented individuals in other professions/sports have ‘off’ days.

    On the other hand, the second and ‘true’ MDC sets massive odds against chance, requiring close to perfection in a one-off test. Aas I say in the article – so I’m repeating myself – is it very surprising that someone who was a talented medium (or successful business person, as the case may be) would avoid being tested at million to one odds, given the negative publicity and repercussions that would result from a failure? I have no idea of the skills (or not) of John Edward. But I certainly don’t think any less of him or his alleged talents for not taking on the MDC. It’s the intelligent course of action, and again, another reason why the MDC is worthless.

    “Your meta-analysis example is completely irrelevant to the sorts of people actually being tested by JREF”

    I’m unsure of what you mean here. Do you mean the deluded individuals who actually apply (in which applying is almost the first failure of a discerning, intelligent mind)? Or are you saying that parapsychology’s allegedly positive results (Ganzfeld etc.) aren’t included in the MDC? Randi would seem to disagree on the latter, given that he’s challenged Radin (http://www.randi.org/jr/2006-09/092206bad.html#i8).

    “If sylvia browne had a success rate less than 10% above chance then she should not be directing police resources.”

    I agree completely.

    “The claim about not trusting Randi is also without merit. He does not participate in the tests unless requested to do so by the claimant. If you don’t like Randi, get someone else to do the test.”

    Given that Randi dishonestly misrepresents facts to his liking (and I know that for a fact), and that his testing panels are usually made up of aligned individuals (eg. CSICOP members), I don’ t think it’s without merit. To be clear though – I’m not saying Randi will manipulate results. I’m saying that mediums and other possible claimants would be justified in being suspicious of anything he is attached to.

    “That’s all I have gotten through so far, but it does not look promising.”

    I don’t see how what you’ve said negates the central points of the article. In short: The MDC offers no true scientific test of a claimant’s alleged abilities, and its unlikely to be an attractive proposition for any intelligent claimant who did have genuinely mysterious abilities. As such it’s worthless, and undeserving of its reputation. It offers nothing (apart from the chance for some deluded individuals to try and get super-lucky, although their chances would be better with a lottery ticket).

  57. Greg the Goofy Antiscience Guy

    @Greg in Austin: “I did not say that website was “stupid.” I said it was hard for me to read…I also said that if one were to look at websites that promote anti-scientific viewpoints, especially information that has been debunked, one would see a common theme in colors and page layout style.”

    Hi Greg in Austin,

    Apologies for implying that you said the website was “stupid”. I was trying to avoid replying individually to multiple individuals with similar viewpoints. While you didn’t say outright that the site was “stupid”, I implied it somewhat when you deviated from criticising the layout/colour scheme and said “I think my head exploded from actually reading the content! I haven’t looked it up, but I’m certain there’s a proportional relationship between the readability of a website and the level of bunk contained therein.” (#33) I guess my wires got crossed…

    You also said:

    “I, and several others, asked a very obvious question which neither Occam nor Greg the Goofy has answered. So, I will ask again, and I will rephrase my question:

    If someone “claims” to have a supernatural ability, and goes to JREF with their claim, and goes thru the entire process of determining a reliable and repeatable method to test that claim, and signs a form that says they agree that the test method is fair, why do they then claim the test was unfair in some way AFTER the test is complete?
    8)”

    My personal opinion is because they are deluding themselves (which I implied in the very first line of my initial post @#38). But my article (and response) wasn’t about people who have willingly entered the challenge, it’s about the MDC being an over-hyped piece of trash. So forgive me for not responding earlier on that point.

  58. @Greg the Goofy Antiscience Guy
    “I don’t see how what you’ve said negates the central points of the article. In short: The MDC offers no true scientific test of a claimant’s alleged abilities, and its unlikely to be an attractive proposition for any intelligent claimant who did have genuinely mysterious abilities. As such it’s worthless, and undeserving of its reputation. It offers nothing (apart from the chance for some deluded individuals to try and get super-lucky, although their chances would be better with a lottery ticket).”

    So your point is that a the reason the high profile woo-masters don’t try is because they may fail? Why isn’t the MDC a true test of the claimant’s alleged abilities? A claim is made and it is tested under controlled conditions. The success level is agreed as true test of the claimants abilities by the claimant. If the odds work out to be 100000 to 1 what of it? The test is for paranormal abilities anything less is noise and the claimant agrees anyway.

  59. Greg the Goofy Antiscience Guy

    @madcap: Thanks for your thoughts. See my previous post to TheBlackCat (#59) for many of the same answers. In addition, I would note that perhaps a more worthwhile “small sample” challenge would be for some smaller wager (whether financial or otherwise) between Randi and selected individuals (to keep the head count down) with a more achievable p-value (given the lack of a million dollars to protect). That would at least be a decent challenge to the John Edward types who earn significant money from their ‘skills’ and therefore should be able to demonstrate reasonably high levels of success in a small sample test.

    Doesn’t make for the greatest copy though, compared to a million dollars. That’s the beauty of the MDC for Randi; it’s publicity, pure and simple.

  60. @Greg the Goofy Antiscience Guy
    “My personal opinion is because they are deluding themselves (which I implied in the very first line of my initial post @#38). But my article (and response) wasn’t about people who have willingly entered the challenge, it’s about the MDC being an over-hyped piece of trash.”

    What other protocol for testing individual claimants can you suggest that would be better than the MDC? Or, are you suggesting that any individual test is worthless?

  61. @Greg the Goofy Antiscience Guy
    “That would at least be a decent challenge to the John Edward types who earn significant money from their ’skills’ and therefore should be able to demonstrate reasonably high levels of success in a small sample test.”

    Because they “earn” money from their “skills” they’re entitled to show less of an ability to prove that ability? Jeebus!

  62. Rift

    @greg the goofy

    You keep calling Randi dishonest and a snake oil salesman. On your website you site the video of Randi watching firewalkers and claiming he was baffled how they did it. (However this was in the ’80s not the 70s). I saw that video and was baffled and intrigued that if Randi was baffled there may be something to it. However after some quick research, there is a physical, non-supernatural explanation. Coal has a low heat capacity and is a good heat insulator, so if you walk fast enough, anybody can firewalk. Obviously Randi discovered this after he said he was baffled and is no longer baffled when he says he has never seen a paranormal event. Firewallking is plain good ole physics at work, nothing more…

    And if you wish to see a skeptic firewalk (and more on the physics)….
    http://www.skepdic.com/firewalk.html

    Like cold reading, firewalking is just a trick. Once you know the secret, you can perform it. Just because at one time Randi stated he didn’t know how it was done doesn’t mean he is ‘dishonest’ but he was actually being very honest. Just because you don’t know how something is done, doesn’t mean it is paranormal.

  63. Rift

    Sorry Greg, You didn’t say that about the firewalkers but was in a comment to your article.

    I was mistaken and I apologize. I read your website several hours ago and it was an honest mistake. But it still stands that the poster of that comment was wrong.

  64. Michael Gray

    (I posted the following entry in the appropriate JREF thread, and reproduce it here for your edification)

    I predicted this very eventuality in the JREF blog.

    Vis: that Ms. Putt would morph from a cheerful, eloquent, intelligent and co-operative ‘good egg’, into a pleading and pathetic whinger**, and that her clear, abject and total failure MUST be blamed on anyone or anything but herself, or that she by malign fiat change the definition of success so that it accords with her profoundly wishful thinking.
    (And that said metamorphosis would be rapid, as is the case with every dowser, doodlebugger or water-witcher to date.)

    The mindset that ‘allows’ one to ignore the reality of one’s prior written contractual agreements in favour of a transparently juvenile but bizarrely creative re-interpretation of plain English, is the same mind-set that engenders the delusion that physical reality exclusively does not apply to one’s-self, and hence the ignorant ‘magical’ approach to life, and the semi-infantile special pleading that occurs as a post hoc excuse.

    Colour me ‘unsurprised’.

    ——————-
    ** This is being as charitable as I am able, given that I care passionately about truth and reality.

  65. brad tittle

    “Plainly Obvious” — Are you nuts, Phil? Expecting anyone to recognize what is plainly obvious is almost as silly as the statement she made.

    That is almost as bad as resorting the the “common sense” line of argument.

    It is “plainly obvious” to me that no one has identified what would nullify the AGW hypothesis. However I suspect that it is equally “plainly obvious” that I am nuts to suggest that in a post about the irrational thought processes of professed psychics.

    Apologies. Hand waving drives me nuts (even though I have to do it all the time).

  66. TheBlackCat

    On the other hand, the second and ‘true’ MDC sets massive odds against chance, requiring close to perfection in a one-off test. Aas I say in the article – so I’m repeating myself – is it very surprising that someone who was a talented medium (or successful business person, as the case may be) would avoid being tested at million to one odds, given the negative publicity and repercussions that would result from a failure? I have no idea of the skills (or not) of John Edward. But I certainly don’t think any less of him or his alleged talents for not taking on the MDC. It’s the intelligent course of action, and again, another reason why the MDC is worthless.

    Claimants are tested at the performance level they claim to be able to perform at. The tests are designed to determine whether the person can do what they claim to be able to do. That is it. I still don’t understand what you think is so terrible and so unfair about that.

    Given that Randi dishonestly misrepresents facts to his liking (and I know that for a fact),

    And you know this how? Sorry if we are not going to take your word for it.

    and that his testing panels are usually made up of aligned individuals (eg. CSICOP members),

    In other words, any testing by a skeptic is automatically invalid? Thanks, that tells us a great deal about where you are approaching this from.

  67. @Dr. Plait

    Just an FYI, when I refreshed the page for this post, the Google Ad to the right came up with an ad for “Psychic Medium MarVeena, Spiritual Readings – www (dot) Marveenameek (dot) com”.

  68. @ Greg the GAGS,
    “…she may just have been having a bad day. The result proves nothing, except for Putt’s ability on that given day.”

    The thing is that when we look back at every challenger, it seems they ALL have bad days on that given day.

    How many “bad days” do they declare to their high-paying clients though? Why aren’t these “bad days” raised as an issue beforehand? Why not during the test? Why not immediately after the test, before the results are known? Or are we to believe that in every case, the MDC resulted in the one and only “bad day” the challengers ever suffered?

  69. Gary Ansorge

    Gee, Greg, I don’t know what her problem was. I hear the Dead all the time(what was that Jerry? Oh, yeah, Jerry says,”Sometimes, S##t happens,,,),,,

    Now, back to my music,,,(which sometimes is so good, IT seems para-normal)(wonder if THAT would be testable???).

    GAry 7

  70. TheBlackCat

    @ AndyD: You may already be aware of this, but the tests often include a test to make sure it isn’t a “bad day”. They very often do an unblinded test first to make sure the powers are working properly before moving on to the blinded test. Of course the powers are working just fine when the subject knows what the outcome should be, but suddenly mysterious interference coincidentally arises as soon as the blinded portion of the test begins. Of course this interference is not detected during the test, as you pointed out it only becomes apparent after the results are in.

  71. rob

    hey, i want to take the challenge.

    i have the ability to pick coin flips correctly!

    so, i propose Randi and Phil get a coin. flip it, and i will amaze them with my predictive prognosticative powers!

    if i get the coin toss correct, i get the $1M and i prove that i gots the Power!!!

    if i get it wrong, i will only admit that the psychic energies weren’t aligned. and the sun was in my eyes. and the coin’s spirit was trapped in the ether than neither Michelson nor Morley could detect so it couldn’t communicate it’s intentions to me. my back hurt. i was cold and hungry and wolves were chasing me. then i will ask to try it again for double or nothing.

    :)

  72. JSug

    @Greg the GAGS:

    The main problem I see with your article is that it starts with the flawed assumption that JREF are trying to disprove the existence of the paranormal. That is not the intent of the JREF challenge, nor would it be possible to do so. It is logically impossible to prove the non-existence of something. The burden of proof is on the one making extraordinary claims, and that is the principle by which the JREF challenge operates. They test *specific* claims made by *specific* persons. If they cannot produce the results they claim they are capable of, they don’t meet the requirements of the challenge. The tests are designed such that success means the claimant was capable of repeatedly beating the odds, not just getting a single result that was statistically significant.

  73. Torbjörn Larsson, OM

    Paranormal perception of astronomers: “Putt off by them” (BA blog).

    Public perception of astronomers: “http://arxiv.org/abs/0905.3956″ (Tweprint/ArXiv on Twitter)

    [Disclaimer: I haven’t read the later. Nor have I been “read” by the former.]

  74. Torbjörn Larsson, OM

    It is logically impossible to prove the non-existence of something.

    That is too bad for logic, since it is done by science all the time. In fact, it is what testing is, disproving facts and theories beyond reasonable doubt.

    Also, there are many explicit theories of universal nonexistence, say of hidden variables or more powerful algorithmic (computable) systems than Turing.

    May I suggest that basic areas of logic are non-overlapping, say predicate logic vs intuitionist logic. So you can’t make universal statements out of such fragile systems.

    The same goes for formal math. It is but when you employ the full power of algorithmic methods, as underlies physics, where you can start to see the underlying monism of natural objects.

    This other problem, that formal systems aren’t powerful enough to comprehend physics, is a show-stopper for philosophy.

  75. TheBlackCat

    That is too bad for logic, since it is done by science all the time. In fact, it is what testing is, disproving facts and theories beyond reasonable doubt.

    The “beyond reasonable doubt” part is where the problems arise. Woo proponents define “reasonable” very differently than most scientists do. There is a difference between disproving beyond reasonable doubt and disproving beyond any doubt whatsoever. The former is what science does, the latter is what many woo proponents demand.

  76. Buzz Parsec

    Hey, the claimant gets to decide what’s paranormal or not? I claim I can win at solitaire on my computer occasionally and that’s a psychic power. Where’s my million bucks? Huh, I’m waiting…

    BTW, if Putt claims she can read anybody at any time, then 5 out of 10 would be an extremely bad day. That’s probably why she agreed to 5 of 10 in the first place.

  77. Torbjörn Larsson, OM

    @ TBC:

    Somehow I’m reading your comment like philosophy is just so much woo. :-o

    I think I like that.

  78. On the subject of debunking – Neal’s Yard is apparently a large chain of “alternative medicine” shops in Britain. They claim homeopathic remedies for malaria, influenza and alternatives to vaccination among other “medicines”. They volunteered to participate in a blog debate about their products, probably thinking it a swell marketing opportunity. The well educated readers of the The guardian.co.uk gave them a thrashing so thorough that they were too frightened to respond at all. It’s really quite brilliant. Not trollish, it’s as if all of your blog readers suddenly went over there to participate.

    http://www.guardian.co.uk/environment/ethicallivingblog/2009/may/26/you-ask-neals-yard-remedies?showallcomments=true

  79. @Shane: “What other protocol for testing individual claimants can you suggest that would be better than the MDC”

    I would think that testing should be a longer term experiment (to enable more realistic benchmarks), without the million dollars (to remove the carnival atmosphere). Kind of like how science works, y’know?

    @TheBlackCat: “Claimants are tested at the performance level they claim to be able to perform at. The tests are designed to determine whether the person can do what they claim to be able to do. That is it. I still don’t understand what you think is so terrible and so unfair about that. ”

    If they agree to the MDC, ala Patricia Putt, then I think they put themselves up as fair game. That doesn’t mean it’s a fair test of whether they have their alleged ability (once more, the short sample size doesn’t allow for an ‘off day’, and the million to one odds on the final challenge are a benchmark far above any reasonable test of ability).

    also @TheBlackCat: “And you know this how? Sorry if we are not going to take your word for it.”

    Apart from others’ anecdotes – and the experience of Rupert Sheldrake mentioned in my article is one public example. (Note also in my follow-up to Randi’s reply, I point out how deftly he manipulates dialogue, such as his thoughts on Dick Bierman’s challenge application.)

    But I know this for a fact from the email exchange I had with Randi after I published my article, where he told me that the ‘errors’ he made in his reply article (which I had asked him to fix) – where he misrepresented the views of a certain person – were intentional, and on the basis of his personal beef with that person. The errors remain, despite my protests.

    But you are right not to trust my opinion, or anyone else’s, when it comes to such things. The sensible thing to do though would be to apply as much critical thinking to the claims of James Randi as to others.

    @TheBlackCat: “In other words, any testing by a skeptic is automatically invalid? Thanks, that tells us a great deal about where you are approaching this from.”

    No, don’t put words in my mouth (and further compound the error by judging where I’m “approaching this from” on the basis of that). I said that having a testing board of aligned individuals would probably make any possible claimants rather suspicious of applying (and thus who could blame a high-profile ‘medium’/’psychic’/etc for not applying). I certainly did not say that “testing by a skeptic is automatically invalid”.

    @AndyD: “The thing is that when we look back at every challenger, it seems they ALL have bad days on that given day.”

    Indeed. But that’s not what I’m arguing at all. My opinion is that anybody who would submit themselves to the MDC has poor judgement and may well be deluded (given the benchmarks, likely failure, and subsequent adverse publicity). I’d say Patricia Putt probably has no psychic talent, if there is such a thing at all. My point though, was that the MDC shows no such thing. All it shows is that Patricia Putt didn’t do very well on that day. That is, it’s useless. Apart from being an oddball challenge where possibly deluded and probably completely unpsychic individuals try and get lucky and win a million dollars in one-off attempts. Booyah, what a WOFTAM.

    In short, I disagree vehemently with Phil that the MDC is the “coolest thing” that the JREF does. At least, I hope it’s not. I’d hope the coolest thing an educational institution does is…educate.

    @Jsug: “The main problem I see with your article is that it starts with the flawed assumption that JREF are trying to disprove the existence of the paranormal. That is not the intent of the JREF challenge, nor would it be possible to do so.”

    That is in fact exactly my point. It can’t disprove the paranormal. It can’t even disprove the alleged ability of a self-labeled psychic. So what is the point of it, and why is it the “coolest thing” the JREF do?

    The fact is though – despite your protestation – that Randi, the JREF, and numerous ill-informed ‘skeptics’ use the MDC as an *implied* disproving of the paranormal (for rhetorical/publicity reasons mostly). See my earlier post where I linked to Randi’s post about Dean Radin (“if the Radin and Sheldrake declarations were really true and properly derived, then they would stand as good evidence for the reality of parapsychology, and would incidentally make the writers eligible for the JREF million-dollar prize. As we know, Sheldrake has directly refused to apply for that prize, and Radin has made the same decision by choosing to ignore it.”)

    “Incidentally”. No, I don’t think that’s how I’d define Randi’s introduction of the MDC at that point in the discussion…
    ;)

    And finally, back to

    @TheBlackCat: “The “beyond reasonable doubt” part is where the problems arise. Woo proponents define “reasonable” very differently than most scientists do. There is a difference between disproving beyond reasonable doubt and disproving beyond any doubt whatsoever. The former is what science does, the latter is what many woo proponents demand.”

    Which returns things to my central point. The MDC doesn’t disprove anything “beyond reasonable doubt”. It’s not science. It has no use apart from being a publicity tool. Overhyped; in fact, an embarrassment to modern skepticism given the reputation it has.

  80. Greg the Goofy Antiscience Guy

    BTW, in case anyone was concerned – I’m not antiscience. Phil apparently thinks I am, having called me that once. I think its rather funny, hence my adoption of it for posting here.

    Oh, and the acronym would be GASG, not GAGS (for AndyD and JSug)…
    ;)

  81. TheBlackCat

    I would think that testing should be a longer term experiment (to enable more realistic benchmarks),

    Why? If these people claim to be able to do this on demand with a very high degree of accuracy, this sort of longer-term experiment is unnecessary.

    without the million dollars (to remove the carnival atmosphere). Kind of like how science works, y’know?

    You can follow the same protocol without actually accepting the money, or even JREF’s involvement.

    once more, the short sample size doesn’t allow for an ‘off day’,

    As I said before, if the subject can have an off day, then the test can be set up to rule that out. The subject can have an unblinded test to make sure there powers are working properly, and then go into the blinded test only if there are not problems. If they are having an off day, the unblinded test would tell them that and they can reschedule. If they can’t tell whether their powers are working or not, then their opinion that they have powers in the first place is totally unreliable, no more reliable than someone pointing at a random person on the street and saying “this person has magical powers because I say so”.

    and the million to one odds on the final challenge are a benchmark far above any reasonable test of ability

    No, it isn’t. The benchmark is set based on their claimed level of performance. How many times do I have to say this. The subject says what he or she is able to achieve, and the tests are set up to see if they really can.

    No, don’t put words in my mouth (and further compound the error by judging where I’m “approaching this from” on the basis of that). I said that having a testing board of aligned individuals would probably make any possible claimants rather suspicious of applying (and thus who could blame a high-profile ‘medium’/’psychic’/etc for not applying). I certainly did not say that “testing by a skeptic is automatically invalid”.

    No, you said they justified in rejecting the million dollar challenge because testing was done by CSICOP members and other “aligned individuals”. Considering the testers often have no affiliation whatsoever with JREF or Randi I can only imagine anyone who is a skeptic is automatically “aligned”. So you are saying they are justified simply because the person doing the test is a skeptic.

    All it shows is that Patricia Putt didn’t do very well on that day. That is, it’s useless. Apart from being an oddball challenge where possibly deluded and probably completely unpsychic individuals try and get lucky and win a million dollars in one-off attempts. Booyah, what a WOFTAM.

    No, it shows a great deal more. It shows that Patricia Putt is incapable of telling whether she actually has powers or not. While the testing was going on she thought everything was going fine and her powers were working well. It was only after the results came in that she started claiming that the protocol was screwing up her powers. That show that she is incapable of telling whether her powers are working or not. If that is the case, then there is no reason to think she has powers ever, since any time she thought her powers were working they could very well have been failing just like they did during the test.

    Which returns things to my central point. The MDC doesn’t disprove anything “beyond reasonable doubt”. It’s not science. It has no use apart from being a publicity tool. Overhyped; in fact, an embarrassment to modern skepticism given the reputation it has.

    Sure it does. It can disprove that the subject can tell between the paranormal and wishful thinking (or can show the subject is a charlatan and a liar, in some cases).

  82. Leander

    @TheBlackCat

    “Sure it does. It can disprove that the subject can tell between the paranormal and wishful thinking”

    On that particular day. Is that what you’d call “beyond reasonable doubt” ? As far as I’m concerned, Greg’s point – as made in that last quote of his you replied to – still stands.

  83. TheBlackCat

    @Leander: If the subject cannot tell whether their powers are working or not on a particular day, why should we believe they work on any day? It is one thing to have a bad day, everyone has them. But if someone cannot tell whether they are having a good day or a bad day until tested, then their claims about what powers they had on days when they weren’t tested are no more reliable than random guesswork.

    It all comes down to this: if someone cannot tell whether their powers are working, then they cannot tell whether they have powers or not in the first place.

  84. Leander

    @TheBlackCat

    “@Leander: If the subject cannot tell whether their powers are working or not on a particular day, why should we believe they work on any day?”

    So let’s say Usain Bolt is running the 100m distance, and think’s he’s outdone his world record. But he’s not. His subjective experience clouded his assertion of his performance. We all know how things can actually feel a lot different from how they actually are. His ability to judge his own performance doesn’t make any statement about the performance measured in the particular example, nor about his performance on any other day. Do you see how fallacious and misleading your attempt to correlate the two is ? I guess that was unintentional though, right ?

    Surely you wouldn’t wanna lead the discussion away from what it ACTUALLY all comes down to: you can never universally disprove anything – be that a person’s ability to assess their performance, or the success of the performance itself. You can only disprove such a thing “beyond reasonable doubt”. And testing said thing on ONE SINGLE DAY doesn’t constitute “beyond reasonable doubt” in any scientifically valuable sense. So the point STILL stands, the MDC is scientifically useless – whether the subjects agreed to it, whether they can assess their performance, whether they achieve the results they claimed they were gonna achieve or not. Plain and simple. That’s pretty poor for “the coolest thing” the JREF is doing. And it leads me to…

    Skeptics like to bring up the MDC – and the fact that noone who entered has successfully completed it, or that many claimants of paranormal gifts don’t enter – as an argument against the existence of paranormal gifts. Now if, as was established here so far, the MDC doesn’t make any scientifically valuable statement – aren’t skeptics who bring up the MDC and said facts as disproof of the existence of paranormal gifts misleading the public ?

    One simple question – yes or no is sufficient, anything else is just plain dodging:
    Shouldn’t anybody TRULY interested in educating (JREF comes to mind) people be interested in refuting these skeptics ? To demonstrate that the MDC is scientifically useless and does produce no scientificillally valauable result at all ? So if none of you here, including Phil, can make a worthy point to prove that the MDC is a scientific contribution to the debate – shouldn’t you be going out to take down those skeptics deluding the public into thinking that the MDC proves or disproves anything ? Yes or no. So simple.

  85. Ciaphas

    I suspect that all the people claiming the MDC was useless would change their tunes if someone actually passed it.

  86. Leander

    @Ciaphas

    Well, I’m not gonna suspect the people who claim that the MDC is useful would do the same in such a case – because I find just throwing in suspicions to be a pretty useless contribution to the discussion.

    And oh, feel free to give that simple yes-or-no-answer to my last paragraph – that wasn’t directed solely at BlackCat.

  87. Rift

    [i]Shouldn’t anybody TRULY interested in educating (JREF comes to mind) people be interested in refuting these skeptics ?[/i] No, but if i say anything else I’ll be dodging.

    [i] To demonstrate that the MDC is scientifically useless and does produce no scientificillally valauable result at all ?[/i] No, but if I qualify it, I’ll be dodging.

    [i] So if none of you here, including Phil, can make a worthy point to prove that the MDC is a scientific contribution to the debate – shouldn’t you be going out to take down those skeptics deluding the public into thinking that the MDC proves or disproves anything ?[/i] No, but again if I explain myself I’ll be dodging…

    Do have anything serious to add to this discussion, Leander?

  88. TheBlackCat

    So let’s say Usain Bolt is running the 100m distance, and think’s he’s outdone his world record. But he’s not. His subjective experience clouded his assertion of his performance. We all know how things can actually feel a lot different from how they actually are. His ability to judge his own performance doesn’t make any statement about the performance measured in the particular example, nor about his performance on any other day. Do you see how fallacious and misleading your attempt to correlate the two is ? I guess that was unintentional though, right ?

    That is not an accurate metaphor.

    First, his world record is measured by objective tests. You are comparing a previous objective test of his performance with a current subjective test. For these people, the JREF is the first objective test. So all we have to go on is previous subjective measures.

    Second, the people can be tested first, going through the whole procedure, but without blinding in place. Then they go through the real test immediately after. And in some cases, I think, they even have a third unblinded test after the real test but before the results are released to make sure their performance is still up at the level they want. So subjects are able to look and see whether their powers are working when they know the correct answer, then be tested without knowing the correct answer, and then be tested again knowing the correct answer. So if they were really using their powers and their powers were not working they would find that out during the initial or final test.

    Third, the purpose of the test is not to determine their maximum level of performance, it is to determine whether they have any performance at all. They are asked what performance level they can perform at consistently, not what their best-ever performance level is, like in your example. The threshold is then set well below that performance level, as we saw in this example, at a level they agree they should have no problem meeting.

    So yes, I suppose it is possible that their powers worked immediately before and after the actual test, in otherwise identical situations, and that they somehow accurately know their performance every other time they use their powers (including just a few minutes before and after) but their ability to assess their performance during the test randomly failed just that one time only and failed in thinking they did well (which is coincidentally what you would expect if they had no powers to begin with). But if you have to go to those sorts of extremes to explain a failure I think you are really grasping at straws.

  89. Greg the Goofy Antiscience Guy

    @TheBlackCat: “It shows that Patricia Putt is incapable of telling whether she actually has powers or not. ”

    Fair point.

    @TheBlackCat:”the purpose of the test is not to determine their maximum level of performance, it is to determine whether they have any performance at all. They are asked what performance level they can perform at consistently, not what their best-ever performance level is, like in your example. The threshold is then set well below that performance level, as we saw in this example, at a level they agree they should have no problem meeting.”

    Can you tell me where I can find more information on this? Everyone I have talked to said there was an arbitrary 1000-1 benchmark for the preliminary, and million to 1 for the final challenge (although it was slightly negotiable, depending on the protocol). This seems to have been confirmed in Randi’s JREF newsletter reply to my original article, in which he discussed the odds with his resident statistician. The statistician replied:

    “Setting the bar for significance is “merely” a matter of deciding how risk-tolerant you’re willing to be. I believe that it is entirely sensible to set a high bar for the $1M prize. Maybe one out of a million is a bit extreme, but it’s your money and your risk.

    On the other hand, you might consider a lower bar for the preliminary test and still protect yourself overall. For instance, you could use .01 (which is frequently seen in the scientific and statistical literature) for the preliminary, and a 1 out of 100,000 rule for the final test – and taken together, you’d know there was only a one-in-a-million shot that someone could get lucky on both.”

  90. Leander

    @Rift

    “Do have anything serious to add to this discussion, Leander?”

    What I’m adding to the discussion can’t be as bad as you make it look, since we seem to agree on at least some things – I asked the yes-no-question you replied to after specifying “if, as was established here so far, the MDC doesn’t make any scientifically valuable statement”. Instead of objecting to this specification you chose to answer the question I derived from it, so I think it’s fair to assume that we agree on the specification.

    Your answer seems to imply though that I’m oversimplifying the situation, and that you somehow think that it’s okay for people interested in educating the public about science and scientific thinking to use unscientific exercises like the MDC to back up their arguments. I personally still think that noone, whether they call themselves skeptics, psychics, ufologists or whatever should use anything devoid of scientifc value to back up their position, and worse, make things that don’t have that value look as if it does. It’s still that simple to me, but maybe I’m having a dull week. If YOU have something serious to add to this discussion, this would be a good time to enlighten me on the complexities of why skeptics should not oppose that kind of behaviour in skeptics, only in everybody else. I’d think ESPECIALLY people claiming an interest in spreading rational thought should be taken to task for stuff like that, and the double standard it involves. But I hope you’ll clear it all up for me.

  91. Leander

    @TheBlackCat

    Okay, I’m sorry…I’m trying to get my thoughts as clear and straight as possible (which, granted, might not always be successful since English is not even my first language), and I’m expecting the people I’m debating with to do the same. Of course we all have bad days, but – no offense – you’ve repeatedly made a mess out of this debate. I’m not in a place to say whether that’s intentional or not. Let me explain.

    In comment #86 you claimed the MDC could disprove someone’s ability to assess their own performance. In #87 I objected and said it could only do on that particular day, a feat that has no scientific meaning whatsoever. In #88, instead of simply agreeing with or objecting to that statement, you totally leave that point aside and try to make a new one – that someone’s ability to assess their performance somehow makes a statement about whether the performance is possible or not at all. In #89 I use a simple metaphor (Putt thought she was doing well, it was objectively shown she wasn’t – Usain thought he was doing well, it was objectively shown he wasn’t – though he in fact could do so, same might be true for Putt) to show the fallacy of your new point. Again then, in #93, you start a whole new set of points, instead of replying directly to what was said. Your first point…that’s not even what the metaphor was about, and it should be obvious. Second point…why all these words to explain something completely irrelevant to our discussion ? Not a single word there explains why you would claim that the MDC disproves someone’s ability to assess their performance (or any ability), or why your point made in #88 is not fallacious. Third point, same problem as with your first point…the world record mentioned in the metaphor is completely irrelevant to the metaphor, and you go on and make lots of words about “peak performance” etc., when that doesn’t have anything to do with our discussion AT ALL. You miss or choose to ignore the most obvious meaning of the metaphor and reply to elements of it that one would have to go out and look for. And your last paragraph describes the situation in a way I never did, so your conclusion that I’m grasping at straws to explain someone’s failure is technically nothing but a cute little strawman argument.

    So I don’t know, if you wanna keep messing this discussion up and spiralling out into areas that were never even the topic, by all means, keep doing so, but I’m done then. Or you give some clear replies to our initial topic – that the MDC can’t “disprove” anything, and is thus scientifically meaningless.

  92. Leander

    @Greg the Goofy Antiscience Guy

    @TheBlackCat: “It shows that Patricia Putt is incapable of telling whether she actually has powers or not. ”

    Fair point.

    Even that isn’t a fair point for the MDC I’m afraid. It’s like concluding Usain Bolt can’t tell whether he’s running or not, just because he overestimated the speed with which he was running.

    Like TheBlackCat said “While the testing was going on she thought everything was going fine and her powers were working well.”

    The “working well” part is crucial. That’s what she was wrong about. Whether she was wrong about the working part, we can’t tell. From that doesn’t logically follow, that if, hypothetically, her gifts exist, she can’t tell whether they’re working at all, i.e. whether she has powers.

    Okay, probably I’m obsessing over this whole debate way too much and should just let it go – but I’m flawed like that, so what the hell.

    @TheBlackCat

    I’m still wondering though why you brought the issue of her being able to make correct statements from a subjective viewpoint about her gifts at all. I guess I’m seeing the point you’re trying to make, but I’m still at a loss as to what that point says about the MDC’s ability to come to scientifically valuable results – which is what was talked about here in the first place. Maybe I should’ve asked you that right from the start, and things would have run a less messy course.

  93. TheBlackCat

    Can you tell me where I can find more information on this? Everyone I have talked to said there was an arbitrary 1000-1 benchmark for the preliminary, and million to 1 for the final challenge (although it was slightly negotiable, depending on the protocol). This seems to have been confirmed in Randi’s JREF newsletter reply to my original article, in which he discussed the odds with his resident statistician. The statistician replied:…

    That is not what I mean when I say performance. When you say, for instance, that you are normally right 90% of the time, but in a test you are right 45% of the time, you are operating at half your normal performance level.

    In comment #86 you claimed the MDC could disprove someone’s ability to assess their own performance. In #87 I objected and said it could only do on that particular day, a feat that has no scientific meaning whatsoever. In #88, instead of simply agreeing with or objecting to that statement, you totally leave that point aside and try to make a new one – that someone’s ability to assess their performance somehow makes a statement about whether the performance is possible or not at all.

    I am not exactly sure how you came to that conclusion about post 88. To repeat what I said in that post, if someone is unable to assess their performance on a given day, then their assessments on other days cannot be trusted. I said nothing whatsoever about whether they can perform or not.

    In #89 I use a simple metaphor (Putt thought she was doing well, it was objectively shown she wasn’t – Usain thought he was doing well, it was objectively shown he wasn’t – though he in fact could do so, same might be true for Putt) to show the fallacy of your new point. Again then, in #93, you start a whole new set of points, instead of replying directly to what was said.

    I did reply directly to what you said. The first thing I do in post 93 is explain why your metaphor is not valid. It doesn’t matter how clear or simple your metaphor is, if it doesn’t resemble the subject being discussed in the important details then it is not relevant.

    Your first point…that’s not even what the metaphor was about, and it should be obvious.

    But this is a critical distinction. A runner’s record is not based on their subjective estimate. Even if they were wrong about their best time, the objective measure of their best time still stands. If all we had from a runner is subjective measures of his or her times, and we know that subjective measure is way off from the objective measure, that means we cannot trust any of their subjective times. They may have been right at some point, but since we know they are not reliable none of that can be trusted. To use the cliche, you are comparing apples to oranges, and then you complain when I try to point out that they are different.

    Second point…why all these words to explain something completely irrelevant to our discussion ? Not a single word there explains why you would claim that the MDC disproves someone’s ability to assess their performance (or any ability), or why your point made in #88 is not fallacious.

    Why not? If someone thinks their assessment may be bad on certain days, they can test their assessment to make sure it is working that day. Built into the testing system is a way to address your concern, to make sure they are not having a bad day in their assessments.

    Third point, same problem as with your first point…the world record mentioned in the metaphor is completely irrelevant to the metaphor, and you go on and make lots of words about “peak performance” etc., when that doesn’t have anything to do with our discussion AT ALL.

    No, it is critical to the metaphor. People are not asked to operate at their peak performance for JREF, they are asked to operate at a performance level they are sure they will be able to perform at, at a level they consider to be a definitive test of their ability to perform at all. If the subject has any sense this would be set at or below their minimum performance level. Yet they are not even able to tell they are performing below what they think is their minimum performance. There is a huge difference between being able to tell whether you beat your world record and being able to tell whether you performed at all.

    So to summarize the problems with your metaphor

    1. Your metaphor assumes that we already know that the subject is able to do the task at all, something we do not know for JREF. It also assumes we have some objective standard for the subject to compare against, which we don’t for JREF.
    2. Subjects are able to test their assessment of their abilities, so they have no grounds for saying their assessment was not working that day.
    3. There is a big difference between being able to tell whether you performed your best ever and being able to tell whether you performed at all (based on your own definition of what is considered “performing”).

    I’ll admit that point 1 on its own is not useful, but the points are not meant to be taken individually. I guess it was mistake for assuming you would see how they all fit together. Let me spell it out for you: The subject is not able to correctly assess their ability to do the task at all today (3). The subject is not having a bad day or does not have bad days (2). The only reason we have any reason to believe the subject can do the task is the subject’s assessment of their ability (1). The logical conclusion that we should not believe the subject’s claim that they can do the task.

    And your last paragraph describes the situation in a way I never did, so your conclusion that I’m grasping at straws to explain someone’s failure is technically nothing but a cute little strawman argument.

    What, specifically, is different from your claim?

    Or you give some clear replies to our initial topic – that the MDC can’t “disprove” anything, and is thus scientifically meaningless.

    Yes, it can disprove that the subject is able to assess their powers.

    I’m still wondering though why you brought the issue of her being able to make correct statements from a subjective viewpoint about her gifts at all.

    Because all we have to go on is the subject’s ability “make correct statements from a subjective viewpoint about her gifts”. That is the only evidence that these people have. If they can’t do that then there is no reason to think they have powers at all. That doesn’t disprove that they have powers, I never claimed it did. But it does mean there is no more reason to think the subject has powers then that my computer is really a shape-shifting gnome, or any other random claim. Occam’s razor comes into play.

  94. Alex

    Okay, having just read through most, if not quite all, of the above debate, I think I might be able to help a bit here.
    The main problem here seems to be some confusion about some of the facts surrounding psychic abilities etc.

    Point 1. (and probably most important) Psychic powers DON’T ACTUALLY EXIST!
    – sorry to inform those of you who thought they did, but along with magic, unicorns, and dragons, psychic powers are just a made up thing. They are mostly used to extract money and attention from people who aren’t aware of this point.

    Point 2. In light of 1., the MDC results will probably make a lot more sense, given that it is simply confirming what the rest of us already knew. The so-called “psychics” are in fact either a: lying to you, or b: insane!

    Point 3. Arguing on the internets about it is not getting you anywhere, partly because the people you are arguing with were already aware of point 1, and partly because that’s just the nature of arguing on the internets.

    Point 4. Has anyone heard “Blue Garden” the first album by “Masters of Reality” lately? I just dusted off my copy and had a listen, and dang! I gotta tell ya, it’s really stood the test of time. Highly recommended if you haven’t heard it before.

    Remember: “Reality is that which, when you stop believing in it, still exists”

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

ADVERTISEMENT

See More

ADVERTISEMENT
Collapse bottom bar
+

Login to your Account

X
E-mail address:
Password:
Remember me
Forgot your password?
No problem. Click here to have it e-mailed to you.

Not Registered Yet?

Register now for FREE. Registration only takes a few minutes to complete. Register now »