Dysfunctional science: My story in tomorrow's New York Times

By Carl Zimmer | April 16, 2012 3:34 pm

In tomorrow’s New York Times, I’ve got a long story about a growing sense among scientists that science itself is getting dysfunctional. For them, the clearest sign of this dysfunction is the growing rate of retractions of scientific papers, either due to errors or due to misconduct. But retractions represent just the most obvious symptom of deep institutional problems with how science is done these days–how projects get funded, how scientists find jobs, and how they keep labs up and running.

Along with the main story, I also wrote a sidebar about how hard it is to hear from the scientists who write retracted papers. I also spoke on the latest Timescast video, which I’ve embedded below. I show up with Arturo Casadevall at about 4:30. There are also lots of links in my story to the original papers. And if you don’t already read it, be sure to check out the blog Retraction Watch, which has been digging deep into the retraction story for years now.

Comments (12)

  1. Hey Carl, What is the evidence that there is a “growing sense among scientists that science itself is getting dysfunctional.” Have you measured the frequency of scientists that believe this to subtantiate the claim?

    [CZ: Seth–I talked to a number of scientists (some quoted, some not) about the arguments presented by Casadevall and Fang and found general agreement. There have also been some striking surveys of attitudes of young scientists pointing in this direction.]

  2. 220mya

    The graph of rising retractions, etc in the NY Times article is rather alarming, but don’t you need to normalize by the total number of papers being published? It may be that the rise in retractions is simply a function of the rise in number of publications, but the proportion of retractions relative to total number of published papers hasn’t changed. I know the Nature study you cited did this normalization and still found a rising rate, but was there any similar info in the Journal of Medical Ethics source?

  3. Mary

    I think that’s certainly part of the current problem. But another piece of it is due to questionable journals.

    It used to be you could say to cranks: “Was that from a peer-reviewed source?” And the answer was always “no”. But now cranks and folks with agendas can get stuff published by all sorts of sub-par providers, which claim peer-review, but don’t actually seem to know what that means. Or if your peers are other homeopaths…well….

    It is increasingly murky, and this doesn’t help us when we want to stand on data.

  4. Here’s a potted summary of the Casadevall and Fang paper highlighted in Carl’s article. There’s a point by point survey at the bottom through which you can express your view:

    http://thermaltoy.wordpress.com/2012/04/01/hidden-gem-iii-whats-wrong-with-science/

    The first 100 responses (before the NYT article came out) indicated very strong agreement with C&F’s argument; you can find the results on the same site. This is an anonymous survey without much in the way of demographics, but we found many consistent responses (mainly relating to issues around career structure) in Science is Vital’s 2011 survey of ~700 UK scientists at all levels. I won’t provide the link to our report for fear of link spam, but you can find it at scienceisvital.org.uk.

    Delighted that these authors spoke up so clearly – we can fix these problems if we act soon.

  5. possibly also worth noting the incredibly increasing complexity of so many fields… and with increasing micro-specialization on the part of scientists it can be difficult to truly know all aspects pertinent to even your own field of study. Finally, science has always been far more imperfect than perceived by layfolks; today the means exist to splash the flaws across the media.

  6. Tim

    Young scientist (astronomer) here. I think that a major aspect of this problem is the rigid hierarchy of journals. In many fields, people submit their research to a “reach” publication, expecting it to be rejected, and work their way down the line. The volume of papers is too high to assess a lot of work on its merit, so people use the journal (or worse, the journal’s impact factor) as a proxy. This feeds an enormous volume of submissions to the glamor journals like Nature and Science. In my field, we often refer to Nature derisively as the Journal of Incorrect Results. Nature is a for-profit journal out to sell magazines, and its articles consigns the details needed to reproduce a result (you know, the science) to difficult-to-find online-only material.

    We are comparatively lucky in astronomy. Almost everyone uses the arxiv and submits to one of a few journals, all of which (except for the glamor mags) have very high acceptance rates. That means more time doing science and less time playing games. Much of the peer-review happens after publication. There are plenty of bad papers, but the field is small enough that an individual researcher can sort through most of the relevant work.

  7. M.

    Carl – as a scientist myself, I can attest that there IS a growing unease at how much time and effort has to be directed at obtaining funding.

    But the retraction thing? Yes, there is an increase in retractions… from about 0.02% to 0.035%. We shouldn’t ignore this trend, and we should try to make it better, of course. But a 0.035% failure rate seems pretty damn good to me.

    What a lot of scientists ARE worried about deeply is seemingly growing distrust, even hatred, of science in the general public. Global warming denialism is doing what creationism never quite achieved.

    Which is why we need to be judicious at wording and titling pieces like this one. Unless someone digs for numbers, it appears as if there is this huge number of retractions and failed research projects, when the actual number is actually really, really small.

  8. robert lindsay

    Take a look at

    Bruce Charlton – The Story of Real Science http://bit.ly/ISb8XT

    I found it extremely insightful and agree with many of his observations

  9. Robert Smith, the former long-time editor of the British Medical Journal has a chilling book on this issue, “The Trouble with Medical Journals.” This is a growing and seemingly intractable problem:

    http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1383755/?tool=pmcentrez

    Amazon.com: The Trouble with Medical Journals (9781853156731): Richard Smith: Books http://amzn.to/IscSEC

  10. Alexis

    In the NYT article, you show a graph with three categories of problems: misconduct/ fraud, errors, and other.
    I get the first one — its made up data and plagiarism. I assume the second one covers mistakes in reporting (type-os in the data, wrong graph given the analysis, etc…) and sloppiness (which is much worse than an honest mistakes). But what is ‘other’?

  11. Al Cibiades

    I wonder if you classify the notorious “super-luminal neutrino” paper as a “retraction”?
    Wasn’t it published with the admonition: It flies in the face of accepted science, but in all scientific honesty, we must report it.
    And they did, to their much (unacknowledged) credit.
    And as a result a better understanding of the approaches and dangers of measurements was gained, and a further proof of Einstein’s assumptions was demonstrated.
    I thought that was the process of Science!

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

The Loom

A blog about life, past and future. Written by DISCOVER contributing editor and columnist Carl Zimmer.

About Carl Zimmer

Carl Zimmer writes about science regularly for The New York Times and magazines such as DISCOVER, which also hosts his blog, The LoomHe is the author of 12 books, the most recent of which is Science Ink: Tattoos of the Science Obsessed.

ADVERTISEMENT

See More

ADVERTISEMENT
Collapse bottom bar
+

Login to your Account

X
E-mail address:
Password:
Remember me
Forgot your password?
No problem. Click here to have it e-mailed to you.

Not Registered Yet?

Register now for FREE. Registration only takes a few minutes to complete. Register now »