Beyond peer-reviewed journals?

By Razib Khan | July 13, 2011 10:12 pm

Joseph K. Pickrell has a provocative post over at Genomes Unzipped, Why publish science in peer-reviewed journals?:

The recent announcement of a new journal sponsored by the Howard Hughes Medical Institute, the Max Planck Society, and the Wellcome Trust generated a bit of discussion about the issues in the scientific publishing process it is designed to address—arbitrary editorial decisions, slow and unhelpful peer review, and so on. Left unanswered, however, is a more fundamental question: why do we publish scientific articles in peer-reviewed journals to begin with? What value does the existence of these journals add? In this post, I will argue that cutting journals out of scientific publishing to a large extent would be unconditionally a good thing, and that the only thing keeping this from happening is the absence of a “killer app”.

[emphasis in the original]


This reminds of the discussion between Melody Dye and Jason Goldman on bloggingheads.tv. After reading Mr. Pickrell’s case, and generally slouching toward a more “open science” stance overall since that diavlog, I think I am now moving toward Melody’s position of “end it,” rather than Jason’s position of “mend it.”

The action in the Genomes Unzipped post is really in the comments, where Mr. Pickrell’ is mixing it up with supporters and detractors. I highly recommend that you check it out.

My own thoughts moving from specificity to generality:

- Mr. Pickrell notes that despite PLoS’s attempts to facilitate comments and interactions, there’s a clear “empty restaurant” syndrome at work. It reminds me of Chris Surridge’s lament in January of 2007 on the Official PLoS Blog about this problem, contrasting it with the vibrant discussion at Gene Expression Classic. I don’t think anyone can deny that PLoS “get’s it” on a deep and fundamental level, but over four years on it still seems that the loci of public discussion is distributed across weblogs and not around the papers themselves.

- Much of the debate is really only relevant to biological science (ergo, PLoS’s biological focus). Physics has arXiv and the social sciences have SSRN. There are many fields where getting the paper in the journal is more of a formality, a stamp of approval which occurs after scholarly consensus has weighed in. Many of the objections to Mr. Pickrell’s broader argument might be examined in light of the outcomes in these other sciences which have moved much further along the direction he suggests.

- Metrics by their nature are going to be “gamed.” I have no doubt of it. The only thing I would offer is that it often takes time for a new equilibrium to be attained, where selfish actors dominate the landscape.

CATEGORIZED UNDER: Science
MORE ABOUT: Peer-review
  • http://www.genomesunzipped.org Joe Pickrell

    Dude, call me Joe :)

    From the discussion at GNZ, I see that these sorts of ideas are pretty self-evident to the physics/comp. sci. crowd, but scary for biologists. I have little doubt this will eventually change, but it might take a while (and I’m convinced a really good app for filtering papers would speed things along).

  • bob sykes

    Peer-reviewed journals are kept alive by the promotion and tenure process. In most, if not all, science and engineering departments today, a candidate cannot be tenured and/or promoted unless he has numerous publications in high-impact, peer-reviewed journals.

    I served on my department’s P&T committee for many years and chaired it for several. In this day of extreme specialization, P&T committees (and the broader faculty) are reluctant to judge the quality of papers they do not fully understand. Peer-reviewed journals are believed to provide quality evaluations by experts in the various specializations, and so committees and faculties defer to the judgements of the referees and editors.

    It is even more extreme than that. In my experience, many faculty and even some P&T committee members do not read a candidates published work. Instead they rely on the existence of the journal papers and personal interaction with and department gossip about the candidate. (I cannot express how frustrated I often was when confronted with this nonsense.)

    I do not believe peer-reviewed journals will lose their importance until tenure disappears from our universities.

  • miko

    bob, don’t forget hiring. As a postdoc in the current job market, the only thing that puts you in the running for jobs worth having is a C/N/S paper (or two). I could publish a cure for cancer in PLoSONE and it’d be GFY from hiring committees. As someone who believes entirely in PLoSONE and hates the culture of the glamour mags, I also have to admit the exultant feeling of getting positive reviews back from one of those journals and the relief of suddenly having career prospects. It’s an ugly, ugly system.

    I don’t think getting rid of tenure is the only answer. Two other immovable objects we could consider trying to remove are intellectual cowardice and laziness.

  • DK

    Peer review is wasteful and evil. The sooner it dies, the better it will be for modern science. (I am saying that as someone who spent countless hours trying to write good reviews).

  • Antonio

    I personally find they way blogs like yours or Dieneke’s present scientific finding and promote lively discussions quite exciting. I am particular keen on the genomics blogs projects because of the non-bureaucratic way the information is presented as well as the opportunities for discussion and interactions among readers and between readers and the authors (bloggers). I wish there were more blogs in my own field presenting new evidence (and revising old ones) in this way. In particular, I don’t think we have to write a full length article, full of “theoretical” blah blah blah just because some model correction or new data changed previous results in an relevant and interesting way. That said the peer-reviewed process has deep roots in academia (in US and elsewhere) and it plays a crucial role with the professional lives of those formally hired at universities, from the hiring in our first job to tenure decisions to funding support (though is true that “better” programs tend to be more flexible and informal while making this decision while less well ranked departments have a more mechanical approach thus making the situation even worse). Thus improvement in the peer-review process can in itself be extremely helpful for science and scientists.

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Gene Expression

This blog is about evolution, genetics, genomics and their interstices. Please beware that comments are aggressively moderated. Uncivil or churlish comments will likely get you banned immediately, so make any contribution count!

About Razib Khan

I have degrees in biology and biochemistry, a passion for genetics, history, and philosophy, and shrimp is my favorite food. In relation to nationality I'm a American Northwesterner, in politics I'm a reactionary, and as for religion I have none (I'm an atheist). If you want to know more, see the links at http://www.razib.com

ADVERTISEMENT

See More

ADVERTISEMENT

RSS Razib’s Pinboard

Edifying books

Collapse bottom bar
+

Login to your Account

X
E-mail address:
Password:
Remember me
Forgot your password?
No problem. Click here to have it e-mailed to you.

Not Registered Yet?

Register now for FREE. Registration only takes a few minutes to complete. Register now »