Nobel laureate Daniel Kahneman calls for 'daisy chain' of psychology replications

By Ed Yong | October 4, 2012 9:05 am

Last Wednesday, Nobel laureate Daniel Kahneman sent an email to a group of a dozen or so psychologists telling them that the credibility of their field was in danger. The recipients all worked on social priming – the study of how subtle unconscious cues can influence our behaviour. It’s an area that has attracted controversy of late, due to failed replications of classic results, the outing of fraudulent researchers, and a more general concern among psychologists about the validity of their field’s results.

Kahneman meant the email as helpful advice, but his wording couldn’t have been stronger:  “Your field is now the poster child for doubts about the integrity of psychological research… I believe that you should collectively do something about this mess.” His solution: a “daisy chain” of replications, where laboratories collaborate to check the results of their neighbours, in an open, transparent, and pre-established way.

Kahneman requested that the email be sent to anyone relevant and, presumably because I have written several pieces on this topic, a copy landed in my Inbox on Wednesday.  I interviewed Kahneman about it, and my story about his challenge (my description, not his) is now up at Nature News.

Here’s some overmatter that didn’t make it into the piece because of length.

  • I quoted Norbert Schwarz in the piece, but he sent over a far lengthier set of comments, which are online in a Google doc.
  • John Bargh, whose name will be familiar to readers of this blog, sent me comments that were cut from the piece during editing. They echo Schwarz’s views in emphasising the “extensive” support for priming effects—“If people continue to read just critiques and calls to action, the perception of a replication problem will probably continue to grow.  I would urge instead a careful reading of the literature as a whole.” But he is also broadly supportive of Kahneman’s suggestion: “I do not see any harm that could be done if social cognition researchers began to replicate each other’s new findings, which is what Danny called for… I support Danny’s suggestion or some similar mechanism that helps to accomplish this important goal of further establishing the reliability of new effects in social psychology.”
  • David Funder, the President-Elect of the SPSP (Society for Personality and Social Psychology) thinks that Kahneman’s email carries a lot of weight with it. “He must be the most prestigious living psychologist. He’s our one and only living Nobel laureate and has a huge amount of respect from just about everybody. He’s just trying to play referee, and telling people to dial down the anger and defensiveness. One of the things that has made me sad about how this has played out is that this should be a scientific discussion and not an argument. For scientists, it should be that the data is the data, and let the chips fall where they may.”
  • In his email, and in a phone interview with me, Kahneman said that priming effects are very subtle, and could be undermined by small changes to experimental protocols at the hands of unskilled experimenters. “Replication is intended as cloning but there are mutations and they can be lethal,” he said. This is why his “daisy chain” idea involves people within the priming field replicating each others’ studies. But social psychologist Brian Nosek makes this good point: “People say there is nuance here, but that’s a hypothesis. That’s not true prima facie. I’d love to see collaborations across laboratories and to see if [expertise] really matters.”We’d then learn not just if results are true, but why they turn up in some labs and not others.
  • Kahneman mentions recently outed fraudsters in his email – these would include Diederik Stapel from last year, and Dirk Smeesters and Lawrence Sanna this year, all of whom used priming techniques in their experiments. I asked him about this, and the effect on the priming field. “It is a feature of priming studies that they’re surprising; some of them have been quite exuberant in the kind of things they have shown,” he says. “It doesn’t follow that people who are doing priming are committing fraud; it’s more likely that people who are committing fraud are drawn to that field. It’s important not to invert the logic of that.”
CATEGORIZED UNDER: Neuroscience and psychology

Comments (5)

  1. There’s a fascinating but almost silent meta-conversation going on here around and about the question of “priming effects” in the practice of science itself, most obvious in David Funder’s contention that “For scientists, it should be that the data is the data, and let the chips fall where they may.” While this is superficially rational, it is a long way from reasonable: his “should” has only a wistful and tenuous relationship with what *is.* Fortunately for the health of science, Kahneman ascribes no superhuman objectivity to the fallible individuals who practice it; his remarkable sanity is founded on a deep appreciation for the many varieties of irrationality that inform all of our attempts to interpret the “given” facts.

  2. Weza Wunderin

    In addition to showing how their work can be independently replicated, these social-psych priming researchers may need to be a bit more forthcoming about sharing their data:

  3. YH

    Brian Owens said

    Posted on behalf of Daniel Kahneman:

    I write to complain about the irresponsible and damaging title that was affixed to Ed Yong’s piece on October 3. The headline asserts that Nobel laureates challenges … to clean up their act. There is no challenge in my letter, and certainly not a challenge for anyone to “clean up their act.†Instead, I offered friendly advice to colleagues whose work I respect, about an image problem they face and how they might deal with it. The misleading title outraged many of my friends, and probably caused real damage by making it harder for priming researchers to address my suggestion. I would not have expected misleading headlines from “Nature,†and hope you will be kind enough to publish this correction.

    Daniel Kahneman

  4. Claudiu Bandea

    If the presumed letter by Daniel Kahneman posted by Brian Owens in his comment above is real, I wonder why would he ask somebody else to post it for him?

  5. I can confirm that the comment from Kahneman is real. He sent it to me, asking me to send it to Brian to be posted, which I did.

    I can also confirm that while I did not write the headline for the Nature piece, I also think it’s a fair representation of the article. I send Kahneman a link to the piece after it was up. He replied to me saying that he was disappointed that the letter was described as a “challenge” but that he could understand how it was perceived thus. He also said the rest of the report was entirely accurate. He did not mention anything about the “clean up their act” bit of the headline.


Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Not Exactly Rocket Science

Dive into the awe-inspiring, beautiful and quirky world of science news with award-winning writer Ed Yong. No previous experience required.

See More

Collapse bottom bar