Psychologists Throw Open The “File Drawer”

By Neuroskeptic | March 17, 2016 3:43 pm


The ‘file drawer problem’ refers to the fact that in science, many results remain unpublished – especially negative ones. This is a problem because it produces publication bias.

Now, a group of Belgian psychology researchers have decided to make a stand. In a bold move against publication bias, they’ve thrown open their own file drawer. In the new paper, Anthony Lane and colleagues from the Université catholique de Louvain say that they’ve realized that over the years, “our publication portfolio has become less and less representative of our actual findings”. Therefore, they “decided to get these [unpublished] studies out of our drawer and encourage other laboratories to do the same.”

Lane et al.’s research focus is oxytocin, the much-discussed “love hormone”. Their lab has published a number of papers reporting that an intranasal spray of oxytocin alters human behaviour. But they now reveal that they also tried to publish numerous negative findings, yet these null results remain in the file drawer because they weren’t accepted for publication.

Is there a file drawer problem in intranasal oxytocin research? If this is the case, it may also be the case in our laboratory. This paper aims to answer that question, document the extent of the problem, and discuss its implications for intranasal oxytocin research. We present eight studies (including 13 dependent variables overall, assessed through 25 different paradigms) that were performed in our lab from 2009 until 2014 on a total of 453 subjects…

As we will demonstrate below, the results were too often not those expected. Only four studies (most often a part of them) of the eight were submitted for publication, yielding five articles (2, 8, 27, 34, 35). Of these five articles, only one (27) reports a null-finding. We submitted several studies yielding null-findings to different journals (from general interest in psychology to specialized in biological psychology and in psychoenodcrinology) but they were rejected time and time again.

Neuroskeptic readers may remember Lane et al.’s sole published negative study (27), as I blogged about it last year. The authors go on to present the results of all eight oxytocin studies. A meta-analysis of all of the studies finds that oxytocin has no detectable effect: “The aggregated effect size was not reliably different from zero, Cohen’s d = 0.003 (95% CI: -0.10 – 0.10)”. They conclude, in an understated but powerful paragraph:

This large proportion of “unexpected” null-findings raises concerns about the validity of what we know about the influence of intranasal oxytocin on human behaviors and cognition… Our initial enthusiasm on intranasal oxytocin findings has slowly faded away over the years and the studies have turned us from ‘believers’ into ‘skeptics’.

So what?

In my view this is a very important paper, and a brave move by the authors. This kind of revelation of what goes on “behind closed drawers” could be an effective remedy for publication bias. I suspect though that prevention is better than cure, and that the best way to keep the file drawers from filling up in the first place will be to reform the scientific process itself.

That said, a ‘skeptic’ might say that Lane et al. are doing too little, too late. After all, their papers reporting positive effects of oxytocin are still out there – and some of them have been highly cited. If Lane et al. no longer have confidence in those papers, should they retract them?

I don’t think so. If we started expecting scientists to retract papers whenever they changed their minds, I think it would have two effects: slightly more papers would be retracted, and scientists would change their minds a lot less. By publishing these results, Lane et al. have ensured that future meta-analysts will be able to include the full dataset in their calculations. In the long run, this will erase any damage caused by the publication bias.

Hat tip: thanks to Bernard Carroll.

ResearchBlogging.orgAnthony Lane, Olivier Luminet, Gideon Nave and Moïra Mikolajczak (2016). Is there a publication bias in behavioral intranasal oxytocin research on humans? Opening the file drawer of one lab Journal of Neuroendocrinology

  • polistra24

    Meta-analysis is a giant point. I hadn’t thought about it in this context. The meta is claiming to pick up a complete picture when half of the picture is hidden.

    Stats have a major problem with zeroes anyway. I noticed a stat the other day: “The average British household eats 5 grams of liver per week.” Nonsense. Nobody eats 5 grams of liver. It’s impossible to parcel out such a tiny quantity of food. Reality: a few people love liver and eat it by the pound, and a vast majority never touch the stuff. Because the mean doesn’t tell us how much of the picture is zero, it’s useless.

    • OWilson

      Publication bias is real, and statistics are abused.

      As there is no such thing as 1.75 children per family, there is no such thing as average weather on the planetary scale.

      It is hot, cold,wet, dry, calm, windy, stormy, cloudy and sunny depending on where it is measured. The extremes are homogenized into an artificial average.

  • Pingback: Can today’s nasal spray influence yesterday’s sex practices? Non-replication isn’t nearly critical enough: Rejected post | Rejected Posts of D. Mayo()

  • Pingback: Markierungen 03/18/2016 - Snippets()

  • Pingback: Oxytocin research null findings come out of the file drawer #SurfaceYourRealSelf | Surface Your Real Self()

  • Денис Бурчаков

    First thing I thought about, after finishing this inspiring post was: “Negative results are much less fascinating. Why publish them, if nobody cares and therefore no one will cite your journal? They ruin a nice coherent image of science moving forward in a right direction. So why challenge accepted paradigm and ruin this profitable illusion, widely accepted by the public?”. I think that publication bias reflects some mistake in “science self-perception” (forgive me this frivolous term).

    • Bradford

      You are absolutely forgiven. I *AGREE*, that “publication bias” *IS* a *MISTAKE* in “scientific self-perception”. Furthermore, it actually hurts people. Nyet? BTW, serious discussion can never be “frivolous”, IMHO….but it can be enjoyable, and bring a smile to one’s face….

      • Денис Бурчаков

        Well, as one raised in dead serious “cite your comrade” atmosphere of post-Soviet science, I think that discussion should be enjoyable. Otherwise it quickly deteriorates towards some “we are the proponents of ultimate truth and we shall lead you to the bright future” kind of discussion. Smile is a sign of humor, and humor is essential to accept the limits of our knowledge. When I am in doubt, I recall this paper – (check the free PDF and read the Experimental Subjects section). It always makes me smile.

        • Neuroskeptic


  • Pingback: Weekend reads: Publishing hypocrisy; false truths; scientists go rogue - Retraction Watch at Retraction Watch()

  • Neuro_Seance

    The widening of pipelines and loosening of figurative corsets will be a wonderful thing for scientists and everyone else.

    As a reader, all I want is a perfectly detailed method, and the data–big grids of data. I don’t need the introduction, and would prefer to read results sections by numerous parties with access to the data, not just the authors. A free-for-all discussion would be far more interesting than “a) what we hoped these data would say and how one could, with difficulty, convince oneself they said it, and b) if several unlikely things were true, those pesky a$$-backwards associations we found would not suggest that we fail to understand the context of the problem, not to mention the problem itself.”

    I left academia because I hated writing introductions. You had to walk to the library back then, or send an undergrad, you know. I said it was stupid to photocopy references for hours at a stretch when it would all be on our monitors within a few years, and that I wouldn’t be back until it was.

  • Pingback: [LINK] “Psychologists Throw Open The “File Drawer”” | A Bit More Detail()

  • Pingback: “Psychologists Throw Open The ‘File Drawer’” - Mad In America()

  • Pingback: Intranasal Oxytocine, a publication bias openly revealed? – ScientiFix()

  • Pingback: I’ve Got Your Missing Links Right Here (26 March 2016) – Phenomena: Not Exactly Rocket Science()

  • teknowh0re

    Good for them!! Excellent idea. Although….it may cause much confusion. Hmm…shouldnt the “negative findings” be INCLUDED in the full data and assessment of any study?!
    I am concerned to know that papers and magazines and other media are choosing only to publish the findings they like or want. Though i cannot say Im surprised. I would like to think that scientific media outlets and publications would be mature enough and intelligent enough to never selectively publish findings in a way that omits important data and gives a false representation of the outcome. Awful!! =(

  • Pingback: Bring out yer nulls? | My Scholarly Goop()

  • Pingback: ¿Está usted moralmente modificado? El posible dopaje no intencionado de nuestra ética | Fox News()



No brain. No gain.

About Neuroskeptic

Neuroskeptic is a British neuroscientist who takes a skeptical look at his own field, and beyond. His blog offers a look at the latest developments in neuroscience, psychiatry and psychology through a critical lens.


See More

@Neuro_Skeptic on Twitter


Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Collapse bottom bar