It’s getting harder and harder to remember what it was like to write about science in the pre-Web 2.0 days. Back then (i.e., 2004), I’d come across an intriguing paper, I’d interview the authors, I’d get comments–supportive or nasty–from other experts in the field, and then publish an article distilling everything I’d learned. It would take months or years for the authors to follow up on their work or for other scientists to publish their own papers attacking or supporting the original research.
How quaint. Let’s take a look at an experience I had yesterday. I was reading a blog called The Evilutionary Biologist. It’s written by John Dennehy, a scientist who works on bacteria and viruses. A couple weeks ago, I only knew Dennehy as one of the co-authors of a paper on trapping viruses I wrote about for the New York Times. (I’ve met him, too, and he’s actually not the least bit evil.) But then Dennehy got the blog fever, and he is now posting great stuff–new research and classic papers. So here I am finding out about important new papers from the blog of someone who was once, for me, the sort of person I’d write about.
Yesterday Dennehy wrote about a new paper on the evolution of the flagellum, the spinning filament that microbes use to swim. This is a topic of some interest to me, because I’m writing a book about E. coli, and much of what scientists know about the flagellum they’ve gotten from studying E. coli. Some fascinating new work is emerging on how the flagellum evolved, based on surveys of flagellum genes and related genes that build other structures. I wrote about some of this work in November in National Geographic. (E. coli’s flagellum was also a star at the Dover intelligent design trial, because intelligent design advocates claim that it must be the work of a designer.) In the new paper, scientists at Arizona State University the University of Arizona report on searches they’ve made for flagellum genes in the genomes of 41 different species of microbes. They identify a core of genes found in all groups of microbes with flagella, and argue that it was present in their common ancestor. They also argue that these genes are the products of a series of gene duplications, descending from a single ancestral flagellum gene.
So, I do what any science writer does. I read the new paper and looked for some comments. I email Nick Matzke, a co-author of an earlier paper on this topic. He wasn’t impressed. To register his displeasure, he wasn’t content just to send me a grousing email. He blogged at length on Panda’s Thumb. Commenters threw in their own two cents. Meanwhile, another source-turned-blogger, Ryan Gregory (whom I wrote about in an article on dinosaur genomes), wrote about the study as well, to which Larry Moran, himself a blogger as well as University of Toronto biochemist, responded harshly in the comments, saying that the paper should never have been published. (Moran, Matzke, and others complain about the methods the ASU scientists used to identify related genes.)
Now, in the pre-Web 2.0 era, all this to-ing and fro-ing happened all the time. At a packed presentation at a scientific conference, people would stand up during the question period and have at it, or head out to the hallways to continue the arguments. But most of this sort of debate didn’t get far beyond the walls of the conference hall. Science writers like me would try to offer a glimpse into the arguments, but there’s a hard limit to how much we can convey in a thousand-word piece. Any other debate had to get channeled into the glacial flow of scientific publications. Now, as this flagellum exchange makes clear, freewheeling scientific debates can reach a wider audience.
This can potentially be a good thing. It may drive the scientific process forward more efficiently, and it may let non-scientists better understand a crucial part of science. But as it stands, this open debate has some big problems. For one thing, it’s incredibly diffuse–a post here, a comment there. It’s not even really a debate. The authors of the paper itself have not, to my knowledge, responded anywhere to all this. (Admittedly, this has all unfolded in about 24 hours, so perhaps I need to lay off the coffee and wait a while.) Obviously, the blogosphere gets a lot of its strength from its decentralized structure, but it seems to me that productive debate is a lot like life. If you pack a lot of enzymes and DNA and other molecules in a tight package, you get life. Disperse them, and you get a few random reactions. Pack comments about a particular paper in one place, and a real debate can emerge. Disperse them across the blogosphere, and you encourage cheap shots and irrelevant tangents, while good observations go unappreciated.
It’s not as if there hasn’t been a lot of talk about how to make this sort of conversation possible. Science papers could be published in an open-access format and readers could post comments directly to the paper. And in fact, there is such a system in place, called PLOS One. In case you’re not familiar with PLOS (short for Public Library of Science), it’s now a real powerhouse in the world of scientific literature, with a number of high-impact journals. (Full disclosure: I was asked to write an essay for one of their journals.) PLOS One, started in January, takes their philosophy a step further.
What I find striking, however, is how quiet it is over at PLOS One. Check out a few for yourself. My search turned up a lot of papers with no discussion attached. Many others had a few comments such as, “This is a neat paper.” There’s nothing like the tough criticism coming out about the new flagellum paper to be found at PLOS One.
I suspect this situation has come about because scientists as a group are only just becoming comfortable in the blogging environment. (Moran, Gregory and Dennehy are all pretty new to it, for example.) It’s one thing to air your complaints in a small room at the annual meeting of the International Society of Helminthologists. It’s another to post them in a place where all of your colleagues–and anyone else with an Internet connection–can read them. So perhaps in a couple years I’ll revisit this issue and see if indeed the debate really has crossed over into a new incarnation.
And if it does, I’ll have to give some serious thought to what a science writer is supposed to do in such a Brave New World…
Update 4.18: I’ve struck Moran from the newbie list. His blog Sandwalk is relatively new, but he’s been online at talk.origins, etc., for many years. My mistake.
April 17th, 2007 at 1:03 pm
Thanks for the post, Carl. You make some astute observations about the tension between communicating in print and commenting online. I tend to agree that having an open forum will be good, not least because it will allow a glimpse into how scientific data are generated, discussed, critiqued, debated, and ultimately rejected or accepted up to and beyond peer review.
For the record, I have had contact with both Howard Ochman and Nick Matzke about topics of mutual interest before this episode, and I greatly appreciate the work that both of them do (i.e., Howard’s genome research and Nick’s work for NCSE plus his flagellum ideas). And yes, scientific arguments can get heated and I myself have written more than a few (*ahem*, dozen) harsh reviews of manuscripts that were not up to snuff. But you’re right — scientists are not used to the blogging environment and we expect there to be a minimal level of decorum to the criticisms, especially when it is one-sided and in public. Nick’s comments may have wonderful scientific merit (I am not an expert in that area), but stomping on toes on a blog is not how one’s reputation is advanced within the scientific community. I totally support the use of blog as a medium for discussing and debating scientific topics, but it does not follow that all interactions in this venue must be adversarial and aggressive. That it often seems to be so is perhaps a symptom of the fight with anti-evolutionists. Fighting that battle directly is, after all, Nick’s primary job but it is not Howard’s or mine. I would rather not have the tone of that “debate” color the way we interact on issues of real scientific substance. Scientists may not know much about blogging, but bloggers likewise need to learn about the etiquette of scientific engagement.
April 17th, 2007 at 1:15 pm
Regarding the problem of pulling this diffuse conversation together in such a way as to foster productive debate, Vannevar Bush made a useful prediction in “As We May Think”: “There is a new profession of trail blazers, those who find delight in the task of establishing useful trails through the enormous mass of the common record.” Sounds like a job for Science-Writer-Blogger-Person…
April 17th, 2007 at 1:20 pm
Back in 2000, Jon Udell (author of “Practical Internet Groupware”, an early blogger, and all-round ahead-of-the-curve web guy) did a report for Los Alamos National Laboratory titled “Internet Groupware for Scientific Collaboration”. An archival copy is online at http://207.22.26.166/GroupwareReport.html; it makes for very interesting reading, considering what’s happened since.
April 17th, 2007 at 1:21 pm
Funnily enough, I wrote about this very problem last night, in the context of the fr*ming debate. My punchline:
April 17th, 2007 at 2:44 pm
This is why we need aggregators like Postgenomic.In fact this particular paper already has a cluster associated to it in Postgenomic but it did not pick up all the comments about it correctly. Nevertheless this would already be enough to alert the authors to the ongoing discussion.
The main problem here is the difficulty in correctly assigning a blog post to the right paper. Science bloggers are lucky in that articles all have unique identifiers in the form of a digital object identifier (DOI) but there is still no common practice to somehow always identify the post as referring to a paper or group of papers. This would require a little extra effort on our part but it would make the aggregation much easier. For anyone interested in following this up there is already an entry for markup tags in Postgenomic for further reading.
Once the aggregation problem is solved then we have everything in place to get feedback. There are several scripts that change the appearance of pubmed and publisher site to alert us for the existence of blog comments related to an article.
April 17th, 2007 at 4:25 pm
I appreciate the comments and criticisms from Ryan and others. Normally I do not go around throwing bombs like this, but this paper was really far off the mark. The idea that the flagellum evolved by the duplication and diversification of one gene is not supported in the PNAS paper or anything else that has been published, and in fact it is contradicted by previous work and most of the authors’ own results. I wouldn’t even mind if the authors had deliberately argued against the old cooption idea and tried to make a detailed case for the one-gene view, but as far as I can tell they just got there via a muddled overgeneralization from the axial proteins to the whole flagellum.
There are various other issues that are also worrisome, but that is the big one. Because the paper was clearly aimed as a shot at the ID/creationists, and was made Open Access, and pro-evolution people were already taking the hints and talking the paper up, I think it was better to deflate it right off the bat.
So I think this is a special case. Most papers, e.g. at PLoS, don’t have these problems and so they don’t attract much attention either on blogs or the journal comments pages.
April 17th, 2007 at 4:30 pm
Carl, Ochman’s at the University of Arizona, not ASU.
April 17th, 2007 at 6:59 pm
Carl,
One other reason there might not be a lot of comments over at PLoS is because it’s hard enough to keep with the literature as it is. Just ask an editor how difficult it can be to get reviewers of manuscripts. I would be curious to see if the authors you contacted would have blogged about it had you not contacted them.
April 17th, 2007 at 8:27 pm
A further issue with PLoS One was also a part of the failure of the Nature open peer review experiment, in my opinion. The journals don’t have much focus, which means that only a small percentage of the readers will have the expertise to comment on a particular article (beyond “nice paper”). Add to that that a fairly small percentage of readers of anything leave comments, and it’s no surprise to me that there aren’t many comments. I would love to make a substantive comment, but PLoS One has only published 4 chemistry articles in 4 months. I’m not positive it would actually be successful, but I think success would be more likely if comment threads were added to specialist journals.
April 17th, 2007 at 11:48 pm
“I suspect this situation has come about because scientists as a group are only just becoming comfortable in the blogging environment. ”
It could also be that they have other venues for discussion in which they’re more comfortable and which have bigger audiences of familiar participants.
There’s no requirement that people discuss what’s at PLOSone *at* PLOSone, is there?
April 18th, 2007 at 1:03 am
The BMJ has had a rapid reactions section on its web pages for some time. Not everything is as silly as this one, but some of the comments are still very on the ball.
Not being a medic, I haven’t explored how well this works for serious articles, but at least the system is running.
Bob
April 18th, 2007 at 4:45 am
At Molecular Systems Biology, we also had a very low rate of commenting on papers, even though we are open access. We just launched our blog The Seven Stones to try to make the discussion more lively and, importantly, to have the two communication channels–formal peer-reviewed papers and informal blog forum–side-by-side. Perhaps this may resolve some of the “tensions” commented on.
referees is not trivial at all!
I believe the central issues remain as to the lack of time for most scientists to navigate through the blogosphere and, therefore, the need for powerful aggregators (see Pedro’s comment).
BTW, as an editor, I can fully confirm that finding appropriate (=competent, critical, decisive, detailed, fast and fair
April 18th, 2007 at 1:59 pm
We need a court system, but all we’ve got is trial by fire.
Wikipedia seems to be able to arrive at consensus, right? Would that be an appropriate court system?
we also had a very low rate of commenting on papers.
I exepect a comment for each thousand readers. If the community is small, don’t expect many comments. Of course, if the original article is a troll… ymmv. This one per thousand rule of thumb comes from help desks. Most everyone seems to think someone else will notice that it’s broken and will call it in. There seems to be a personality type that comments. Reasonable people adapt themselves to their environment. Unreasonable people adapt their environment to themselves. So progress is due to unreasonable people.
April 18th, 2007 at 7:40 pm
Just recently Dean Esmay (yes, that Dean Esmay) made an observation on commenters on his blog. Namely that of every 100 visitors he got 1 commenter. But read the essay for yourself.
I think you’ll find that it’s a pattern common to every blog out there, with a few notable exceptions. The great majority of the time an author doesn’t draw the sort of person given to voicing his opinion readily. So a popular blog doesn’t get much commentary out of its traffic.
April 18th, 2007 at 10:49 pm
TR Gregory said: “I totally support the use of blog as a medium for discussing and debating scientific topics, but it does not follow that all interactions in this venue must be adversarial and aggressive. That it often seems to be so is perhaps a symptom of the fight with anti-evolutionists.”
I think that we must be prepared for the (agreed, unfortunate) possibility that most, if not all, “interactions in this venue” may be more, ahem, *robust* than we might like. As a layperson interested in science, my hope is that this will not discourage wide, free, public electronic dissemination of scientific papers.
Re the fight with anti-evolutionists as a cause of aggressive interactions, I think aggressive interactions may be found in abundance in the wider blogosphere, in e-mail fora and newsgroups before that, in peer-reviewed scientific journals before that (letters to the editors have been particularly good sources) and in discussions between scientists in various formats going back as far as one wishes (Frazier-Ali had nothing on Hooke-Newton).
April 19th, 2007 at 12:55 pm
Stephen:
Wikipedia’s Featured Article Candidates process is a good example of the sort of disputation arena I’d like to see in more places.
April 20th, 2007 at 12:03 am
Too bad the 2005 ivory-billed woodpecker paper in Science didn’t get this sort of attention.
April 21st, 2007 at 11:49 am
When nature tried to go preprint back last summer I was somehow surprised by the little participation to the comment area of the pre-published papers. I know scientists tend to be shy (especially young ones) and I can imagine few of them will expose themselves in a non-cozy environment such as a publisher website; yet I thought we would find more participation. I thought novelty might be an issue: these things need time; then I asked a physicist friend of mine how things work out in their field where the giant arxiv.org is an everyday reality. He said same applies there: very few people would leave public comment while most would make their remarks privately. At the end of the day this makes sense: it’s easier to tackle the topic this way.
April 21st, 2007 at 6:24 pm
I tempted to just say, “This is a neat paper.”
I don’t have enough expertise to parse the hullaballoo surrounding the original flagellum paper, so I’ll just have to wait for the dust to die down.
But I did appreciate Carl’s backing away one long step and taking an overview at some of the issues raised.
In any event, it’s always fun to note that real scientists may be relied upon to do anything BUT march in the kind of conspiratorial lock-step that the IDiots and Creationists invariably accuse them of.
April 24th, 2007 at 3:44 am
We are attempting to redo the entire portion of the Liu-Ochman study that produced Figure 3 and the homology results that led the authors to the “all flagellar genes from one” conclusion. This will take a little time. In the meantime, here are the results of a preliminary attempt to replicate just the FliC homologies for E. coli, reported in Liu & Ochman’s Figure 3.
May 20th, 2007 at 3:38 pm
I don’t think the lack of chatter at PLoS ONE should be taken as a sign that it’s not working. Like citations, most papers won’t generate significant online discussion. On the other hand, if the flagellum paper had been published at PLoS ONE, a lot of this discussion could have happened there with the added bonus that the paper’s authors would have been more likely to directly respond. I elaborate a bit on this at Biocurious:
http://biocurious.com/give-plos-a-chance
Thanks for the thought provoking post!
April 23rd, 2008 at 8:44 pm
At least in physics, people have tried to set up comment forums at least three times:
http://web.archive.org/web/20030324230036/http://www.quickreviews.org/
http://web.archive.org/web/20051125081307/www.science-advisor.net/
http://web.archive.org/web/20061115051156/http://www.physcomments.org/
All these sites have failed. I suspect there’s little percentage in contributing, if you’re an expert, unless there’s already a lot of other experts contributing, at which point network effects kick in.
Great post and comments!