Yesterday I wrote about the arsenic life saga, prompted by a long retrospective feature by Tom Clynes in Popular Science. While I recommend the piece, I expressed reservations because it passed along the “scientists besieged by bloggers” spin on the events, when the actual history doesn’t support that.
Clynes (whom I’ve never met) emailed me in the evening with this comments, which he allowed me to share:
Carl,
Thanks for your comment on my Popular Science feature on Felisa Wolfe-Simon’s arsenic-life saga. In some ways, I think you’re on target, though I would like to provide a bit of clarification: Throughout the story, when I convey an argument made by someone who’s on one side of the issue or another, it doesn’t mean that I necessarily buy into that argument.
To that end, I’d like to add a bit of context to a paragraph that you quote, regarding the storm of criticism and the paper’s authors going “underground.” You follow the excerpt with your comment that “Clynes has us believe that this barrage of extraordinary, brutal criticism (or perhaps questions from journalists) forced Wolf-Simon and her colleagues to go into witness protection.”
Actually, I don’t believe that, nor would I have my readers believe it. I think it would have been useful to your readers for you to have included my next paragraph, which makes it clear that I am in fact spotlighting both sides of a polarized dialogue regarding this particular point:
Microbiologist Jonathan Eisen of the University of California at Davis called the lack of response “absurd” and told Carl Zimmer from Slate, “They carried out science by press release and press conference. They are now hypocritical if they say that the only response should be in the scientific literature.”
Though I didn’t state my opinion in the story (better for readers to decide for themselves), I will here: I think that Eisen is on the money here.
Some other opinions: Do I think that the arsenic-life paper was flawed? Yes. Do I think it that some of its conclusions will be dissolved by further investigation? Yes. Do I believe that NASA’s hyped-up approach to publicizing what was actually a rather understated paper was ham-handed, and damaging to everyone involved? Big time.
Do I think the paper never should have been published? No. In a profession where young scientists are advised to avoid controversy as they build their careers, Wolfe-Simon pushed against a paradigm and sought answers to some very big questions. She passed through the same peer-review hoops (imperfect as they may be) at Science as other scientists must. Yes, her research was imperfect and yes, she likely overreached—but plenty of scientific papers are flawed, and many young researchers go too far. If scientists aren’t willing to subject themselves to the possibility of failure, science can’t possibly progress.
Critically, there’s nothing to indicate that Wolfe-Simon did anything unethical, which might have justified the shrill tone and sweeping proportions of the response—and the fact that she was singled out among the paper’s 11 authors. True, she was the lead author, and it was her hypothesis. But it’s surprising that Ron Oremland, the lab director and principal investigator, is rarely mentioned in the criticisms.
If my story has a bottom line, it’s in this quote by the University of Colorado’s Alan Townsend: “Absent major ethical violations, no junior scientist full of passion for an idea deserves crucifixion for a professional failure or two. If a paper is flawed, it should be dismissed. The scientist should not.”
October 1st, 2011 at 10:09 am
Clynes make (or at any rate reiterates) a good point here. The “arsenic life” paper is not exceptional. Flawed papers come out all the time; scientists see hundreds of flawed papers. Many are much worse than this one. We rapidly learn never to trust a single paper, or even a single research group no matter how skilled and honest they are. Work has to be replicated before it’s believable.
(One of the comments on the fast neutrino paper was something like, “One paper is an observation. It’s not a discovery until it’s repeated.”)
This may be something that scientists know but that hasn’t been well communicated to the public. Journalists, even the very best, want to talk about new things, which means talking about new work before it’s repeated. So readers are constantly barraged first by “new discoveries”, many of which will quietly fizzle out through lack of replication.
Only some of the fizzles will be reported, because fizzles are rarely news, and because they happen slowly and incrementally. As scientists, we see the fizzles happening constantly, but readers outside of the scientific community — even the most thoughtful and careful reader — only see the fizzles when the original paper is a really big deal and when the replicates come quickly. And so, understandably, they think that this particular work is being singled out, specifically targeted. (The XMRV story is a good example.) Meanwhile, scientists are a little puzzled by the fuss, because this is just one of a hundred ongoing fizzles that we are watching happen.
So: Clynes is correct that this is not unusual. It was an ambitious paper with an interesting idea, probably less well thought through and less well controlled than it could have been. Post-docs who don’t think of important controls are not rare; they’re the majority.
I know that the best science journalists do try to emphasize that science works incrementally and through repetition, but most don’t, and even the best end up reinforcing it because of the emphasis on “new discoveries”. It’s understandable, because the full process is slow, boring, and often negative, so I don’t have any answers. But it would be nice if the public had a better understanding of this.
(Clynes didn’t really address your point that there was not in fact a huge blog outcry [I'm also uncomfortable with his use of the word "shrill" -- would he use that if the major critique was from a male? - but let it go]. You said that there were only two blog responses; I wonder if that might have been magnified by blogs that, like mine, immediately linked back to the critical posts. A google search might well have turned up many blogs mentioning it, all of which were simply linking back to one or two original posts.)
October 1st, 2011 at 10:36 am
It’s been nearly a year. Why hasn’t anyone tried to grow these things and finally settle this debate? Now scientists don’t have to disprove every crackpot theory out there, but the fact that this story won’t go away merits some attention. At the very least they found a bacteria which can survive in a high arsenic environment which should merit some further investigation.
October 1st, 2011 at 10:52 am
@ Chris -
I wouldn’t jump to the conclusions you’ve arrived at. Look at Carl’s original post earlier this week, in which he notes Rosie Redfield’s most damning critique. Here’s the link to her post:
http://rrresearch.fieldofscience.com/2010/12/arsenic-associated-bacteria-nasas.html
October 1st, 2011 at 11:35 am
Clynes says what I wanted to say in my comment on the other post, only written all eloquent-like.
@Chris:
There are a number of scientists trying to grow the arsenic bugs, most prominently Dr. Redfield herself. She’s got a series of posts on her blog covering her ongoing experiments on GFAJ-1, really quite fascinating, and revealing of what real, day-to-day science is like- e.g. “Why isn’t this workinggggg??!”
What’s actually going on with these guys will be revealed, eventually. Good science takes time.
October 1st, 2011 at 11:41 am
@ zmil -
Well said. Good science takes time, but also takes excellent thinking, with respect to the design of experiments and collecting data.
As an aside, Redfield has also debunked yet another NASA science reporting fiasco last year with the claim of yet another “Martian” meteorite containing “bacterial fossils”:
http://rrresearch.fieldofscience.com/2011/03/is-this-claim-of-bacteria-in-meteorite.html
October 1st, 2011 at 11:42 am
“Shrill”? Probably just me but I keep getting the impression that the story here is that the public was witness to the kind of “honest” conversation scientists have with each other and they found it off-putting. Wolfe-Simon’s sin is that she is playing to this public perception rather than standing up for her critics.
October 1st, 2011 at 1:02 pm
@John
Indeed. The one thing I’ve learned, repeatedly, over my brief graduate career- there is no such thing as too many controls. My last two months of lab work would have been unnecessary had I done just one simple control…no regrets, though. Troubleshooting is a good way to really learn the details of a project, and to learn said ‘excellent thinking.’
I remember that bacterial fossil thing- that was a much more ridiculous paper, to my mind. It was like they’d never looked through a microscope before. You’ve got to do better than say, ‘look at this strange shape here’. Seeing weird crap in the microscope is the rule, not the exception.
October 1st, 2011 at 2:53 pm
Nice to read Clynes’s comments, thanks to Carl for posting them and Clynes for allowing them to be shared. It is interesting to see the intentions of the author, to give a fair hearing to both sides of the issue, in light of David Dobbs’s post on Neuron Culture that drew attention to how the piece was received based on comments from the readers.
Also, slightly off topic, but I think it is worthwhile to draw a distinction between the meteorite fossil paper, published in a fringe journal by a NASA engineer with no established authority on bacterial fossils, and the arsenic life story which was published in a high profile journal by a team of authors with a long list of publications in the field. At the organizational level NASA explicitly refused to endorse the former, while putting the full weight of its media apparatus behind the latter.
October 1st, 2011 at 3:24 pm
[...] of the Popular Science piece, or at least one of the talking points of Felisa Wolfe-Simon. This elicited a response from the author of the piece, which Carl posted. Even if you’re not deep into the weeds of the “arsenic life” [...]
October 1st, 2011 at 4:30 pm
I wonder why Tom Clynes is willing to tell us what he thinks here (” Do I believe that NASA’s hyped-up approach to publicizing what was actually a rather understated paper was ham-handed, and damaging to everyone involved? Big time.”) and not in his Popular Science article.
Here, it’s okay for us to know the conclusions he came to by reporting his feature on Felisa Wolfe-Simon’s arsenic-life saga. There, it’s not okay. “Though I didn’t state my opinion in the story (better for readers to decide for themselves), I will here: I think that Eisen is on the money here.”
If it’s better for readers to decide whether NASA and the scientists involved in the paper were making an absurd stand by refusing to respond to public criticism after inviting public scrutiny with a press conference and big build-up, or, on the contrary, doing the scientifically reputable thing by waiting for responses to emerge from the peer-reviewed journals, then why is it better for Popular Science readers to make this decision without knowing that the reporter who talked to all sides in this dispute developed over the course of his reporting a view on that question?
Are we supposed to believe that Popular Science readers who are capable of calmly absorbing both sides of a polarized dialogue and deciding who’s more right suddenly become incapable once the reporter weighs in with his own conclusions? Seems like a stretch, but maybe that’s it.
Or is the logic that if the Popular Science account included what Clynes himself thought this would lessen its credibility with Popular Science readers because it would be obvious that the reporter is biased? This is the sort of reply many so-called mainstream journalists would give. But I think it misses the point of my question. Clynes clearly developed these conclusions by reporting the story. They are not biases he came in with but reasoned outcomes of his own inquiry. Readers interested in science should be handle that… right?
The case for withholding a reporter’s conclusions from readers seems to me to rest more on convention than thought. Clearly, Clynes is comfortable telling the world that he has opinions and letting us know what they are. And the reason is precisely that they came from his reporting. Protecting the readers of Popular Science from those conclusions is an outdated practice. One of the names for this realization is… blogging.
October 1st, 2011 at 4:39 pm
@ neil -
Me thinks NASA should not have launched such a lavish publicity junket on behalf of Oremland, Wolfe-Simon and their colleagues, though NASA isn’t the only one to have made such an event of a supposedly important scientific discovery. I noted at Carl’s previous posting earlier today that this seemed all too similar to the announcement that the primate fossil Darwinius was somehow the “missing link” from lower primates to the Great Apes and humans, which was held at a well publicized venue, the American Museum of Natural History. One of the authors of the scientific paper describing Darwinus later admitted that other systematists were right in noting that, at best, Darwinius was merely a sister taxon based on cladistic analysis.
October 2nd, 2011 at 12:11 am
Alan Townsend has the right of it.
October 2nd, 2011 at 9:20 am
@ David – If the “arsenic life” paper had been promoted solely by Wolfe-Simon, then I might concur. However, NASA invested ample time and other resources to publicize this ground-breaking “discovery”, in much the samme manner that a team of European vertebrate paleontologists and a European natural history museum opted to promote the discovery of the “missing link” between lower primates and the Great Apes (including us), Darwinius, by holding a press conference at the American Museum of Natural History.
Ironically both are examples are consistent with what Chris Mooney and Sheril Kirshenbaum had advocated that scientists do to become effective communicators of science in their “Unscientific America”, but instead became notorious examples of publicizing flawed science, though in Darwinius’ case it was mere hype to coincide with the Darwin bicentennial year, whereas with “arsenic life”, it was something fundamentally far worse, promoting a seriously flawed paper as though it had indeed reported a major scientific discovery.
@ Jay Rosen – I concur completely. What you wrote needed to be said IMHO.
October 2nd, 2011 at 2:05 pm
re #5;
My memory may be off, but I don’t recall NASA being in any way involved in the Hoover paper or its subsequent publicity, beyond the fact that Hoover once worked with NASA in the past.
October 2nd, 2011 at 10:14 pm
@ amphiox -
The author lists his address as a NASA center at the time of publication:
http://journalofcosmology.com/Life100.html
That is indicated as follows:
Fossils of Cyanobacteria in CI1 Carbonaceous Meteorites
Richard B. Hoover, Ph.D. NASA/Marshall Space Flight Center
October 3rd, 2011 at 1:04 pm
Here’s is a passage that is factually incorrect and also plays “the wronged genius” card at several levels: “Do I think the paper never should have been published? No. In a profession where young scientists are advised to avoid controversy as they build their careers, Wolfe-Simon pushed against a paradigm and sought answers to some very big questions.”
1. The Oremland group was never told it should not publish its findings — but that it should have done so after performing basic controls and scaling their conclusions to fit results, not what they would like to think the work showed.
2. Researchers like Penny Boston have been discovering and characterizing bacteria as exotic as GFAJ-1 for literally decades, and they never got the hype (or even recognition) that their work actually merits.
3. All results that challenge paradigms are subjected to extra scrutiny, as they should be. The group’s conclusions were anything but modest and their tone even worse. The term “shrill” is better applied to the original announcement, rather than to the responses that met it.
4. I agree that Oremland has a much larger share of responsibility than he has been willing to shoulder, but he’s older, male and established. So he’s cut his losses and is back to business as usual, leaving others (most notably Rosie Redfield) to clean up the mess. Wolfe-Simon, young, a woman, and just starting, is not as lucky — and giving interviews to Glamour Magazine while stonewalling colleagues over the scientific issues did not help.
5. The funding climate in the sciences is so bleak that long-established researchers with decades of work are being pushed out. To present Wolfe-Simon as a hero rebel “wronged” and “forced out of science” by stodgy sticks-in-the-mud (or jealous wannabes) borders on the disingenuous.
October 3rd, 2011 at 2:29 pm
People (male and female) are often especially shrill when they’re right, as was the case of the establishment scientists smacking down this young upstart who herself might have been seduced by the glamour. It’s unfortunate and unnecessary, although the refutation was necessary.
October 4th, 2011 at 1:08 pm
One more item in Mr. Clynes’ response: “Critically, there’s nothing to indicate that Wolfe-Simon did anything unethical, which might have justified the shrill tone and sweeping proportions of the response.”
Leaving aside the “shrill” part, I want to point out that this was essentially a microbiology paper lacking level 101 controls; not exotic, difficult techniques but such basics as density gradients. If a scientific paper lacks basic controls there are two conclusions: either the authors are incompetent or they did the experiments, did not like the results they got, and decided to suppress them.
If the former, I dread the future of astrobiology; if the latter, I dread the future of science.
October 4th, 2011 at 5:22 pm
Whether the paper was so flawed it should not have been published is not a matter of opinion. The bacteria are obviously growing on the phosphate contaminating the 40mM (!) arsenate being used in the media. I do allege misconduct by the authors. They grossly misrepresent the data in the Supplementary Table 1 in the initial article and the Technical Comments and perform another unacceptable distortion of the statistics (standard deviations) in the Technical Comments.
October 6th, 2011 at 4:45 am
[...] Tom Clynes on arsenic life (The Loom at Discover Blogs, 1st October 2011; Carl Zimmer) [...]
October 18th, 2011 at 3:22 pm
[...] Jay Rosen commenting at DiscoverMagainze.com [...]
October 27th, 2011 at 3:23 pm
@ Saul Davis
The authors measured P in the 40 mM As media, and its concentration was no different than in the no-As condition where growth did not occur. I think that you are grossly misrepresenting…
November 18th, 2011 at 6:13 pm
[...] Zimmer. Oct 1 2011. Tom Clynes on arsenic life. The Loom (Discover). http://blogs.discovermagazine.com/loom/2011/10/01/tom-clynes-on-arsenic-life/ Replicating the [...]