De-discovery round-up (plus a correction)

By Carl Zimmer | June 30, 2011 1:30 pm

It’s been very gratifying to listen to the conversation that’s been triggered by my essay in this Sunday’s New York Times on scientific self-correction. Here, for example, is an essay on the nature of errors in science by physicist Marcelo Gleiser at National Public Radio. Cognitive scientist Jon Brock muses on how to get null results published.

I also got an email from Eliot Smith, the editor of the Journal of Personality and Social Psychology who accepted the controversial clairvoyance paper I described in my essay. I wrote that three teams of scientists failed to replicate the results and that all three studies were rejected by the journal because they don’t accept simple replication studies.

Mr. Zimmer

Your recent Times column stated the following:

Three teams of scientists promptly tried to replicate his [Bem’s] results. All three teams failed. All three teams wrote up their results and submitted them to The Journal of Personality and Social Psychology. And all three teams were rejected — but not because their results were flawed. As the journal’s editor, Eliot Smith, explained to The Psychologist, a British publication, the journal has a longstanding policy of not publishing replication studies. “This policy is not new and is not unique to this journal,” he said.

In fact, JPSP has received only one submission reporting failed replications of Bem’s studies. I did reject that paper based on the reason your column stated.

And to put that in context, I also rejected another submission to the journal that reported successful replications of some of Bem’s studies, on the same grounds.

I believe that a published correction is warranted; the difference between one and three papers is quite meaningful in this context.

Best regards,
Eliot Smith

I’ve passed on Smith’s message to my editor at the Times, and I’ll also take this opporunity here to apologize for the error.

I’m not sure how meaningful it is in the context of my essay, since my point was that policies against publishing replication studies get in the way of science’s self-correction. But a mistake is a mistake.


Comments (4)

  1. One almost wonders if there should be something like “The Journal of Non-Replication” that accepts only papers that failed to replicate results.

  2. As I understand it, there were three unsuccessful replications, which were all described by this one rejected paper. So, in a sense, you’re both right. (As Smith no doubt knows).

  3. jackd

    Would it be feasible, I wonder, for a journal to run a regular (annual?) follow-up issue, which would summarize replication studies of results previously announced in the same journal? The difficulty would be marshaling the resources for review of the replication papers themselves, I suppose.

  4. Jack

    Thanks for the links to the two other articles – quite fascinating (and actually quite worrying) stuff.

    As a layperson I always thought science would replicate as a matter of course. I still believe this happens and quickly too in respect of ‘significant’ perceived threats – but replication of exact methodology – as referred to by Gleiser – does not seem to be insisted upon.

    It is the replication of results in the original research that are the focus – and others will try their own techniques to try and do this – take XMRV for instance.

    Are exact methods and conditions EVER replicated in science – or even attempted I wonder. IS such a thing even possible? Should such attempts even be made?

    After all if through tried and tested techniques the same results cannot be reproduced or are even believed to have been possible in the orginal research (if said research is questionable); then how far should science have to go?

    All very interesting. I should like a follow-up piece if at all possible Mr Zimmer.


Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

The Loom

A blog about life, past and future. Written by DISCOVER contributing editor and columnist Carl Zimmer.

About Carl Zimmer

Carl Zimmer writes about science regularly for The New York Times and magazines such as DISCOVER, which also hosts his blog, The LoomHe is the author of 12 books, the most recent of which is Science Ink: Tattoos of the Science Obsessed.


See More

Collapse bottom bar