PubPeer and Anonymity in Science

By Neuroskeptic | December 12, 2015 10:40 am

 

anonymity_scienceEarlier this week Times Higher Education (THE) published a very interesting piece called Should post-publication peer review be anonymous? It takes the form of an exchange of open letters between the organizers of PubPeer, an online forum which allows scientists to post anonymous comments about any published paper, and Professor Philip Moriarty, a physicist at the University of Nottingham (wh0 has been featured on this blog before).

The PubPeer team open by noting that PubPeer, which positively encourages anonymous submissions, is by far the most active scientific discussion site:

Our experience has been that anonymity greatly facilitates scientific discussion with little if any effect on the quality of comments. The success and growth of PubPeer (we receive approximately 1,000 comments and 500,000 page views a month) can be compared with the activity on other services that forbid anonymity: the absolutely moribund commenting facilities provided by journals and the relatively steady, low-level use of the PubMed Commons system that is offered by the PubMed biomedical archive. In addition, we find that without anonymity, many serious issues are simply never aired – presumably because the competitive and hierarchical environment in research causes most researchers to avoid public criticism of colleagues.

They acknowledge the fear that anonymity facilitates personal attacks and other low quality posts, but they say that PubPeer has systems to prevent this:

Comments on PubPeer must be based upon verifiable information (usually the data in a publication), which focuses discussion on the science. We moderate comments and provide reporting facilities to enforce this guideline.

Moriarty – who has made many contributions to PubPeer himself, under his own name, waiving his anonymity – says that he “regularly sings the praises of PubPeer” because he knows “just how essential [it] can be in highlighting questionable research.” Nonetheless, he goes on to say

I am firmly of the opinion that, too often, online anonymity debases communication… I want to strongly discourage anonymous commenting, to attempt to change the commenting culture so that anonymity is not the norm. But I do not want to ban it… you [PubPeer] are resigned to accepting anonymous commenting as the norm because “the competitive and hierarchical environment in research causes most researchers to avoid public criticism of colleagues”. This is a shocking indictment of our research system.

In other words, Moriarty is deploring the fact that scientists need anonymity. Free discussion, including critique, should be the norm in science, and it’s a shame that scientists should feel they can only speak honestly anonymously. The PubPeers agree with that, and say that

We are hopeful that services such as PubPeer will slowly tip the balance towards the acceptance of open discussion and criticism. For this to happen, the research community will need to stop turning a blind eye to poor-quality work and become much more supportive of those who raise valid concerns. In the meantime, we think it is unfair of opponents of anonymity to demand that PhD students, postdocs or indeed any other scientists take risks that most established researchers, journals and institutions currently avoid.

Moriarty responds, however, by saying that PubPeer commenters post anonymously even when there isn’t any risk to their career. In other words, he worries that anonymity has become a ‘norm’, even when it isn’t necessary. He cites the enormous discussion thread about an anti-anonymity editorial by plant scientist Michael Blatt,

The PubPeer thread on the Blatt editorial is a great example of this [norm] – the vast majority of those commenting are anonymous, yet their comments are innocuous in the context of any type of potential damage to their careers. I will always challenge that norm.

The point here, I think, is that it’s one thing to use anonymity to challenge a consensus, but it is less clear why anyone would want to anonymously defend a popular viewpoint.

The exchange ends there. The THE article ends with an interesting piece by Paul Benneworth. He recalls two harshly negative, anonymous peer reviews he received in the past. But he says “On each occasion, I was stunned at first, but I have to concede that, on reflection, the reviewers were almost certainly right”. He also confesses to writing some acidic reviews himself, and that “the option to write harsh reviews is central to any thriving science system that prizes integrity over system-gaming.”

You can probably guess my stance on all this. I wrote a piece – as my Neuroskeptic pseudonym – for Trends in Cognitive Sciences praising Anonymity in Science, and since then I have argued that Science Needs Vigilantes including anonymous ones. However I don’t believe that anonymous comments should be always given the same weight as signed ones. We should take them with a large pinch of salt if they contain statements that are difficult to verify (rumors, opinions etc.). However, when they make concrete empirical statements, these must be taken seriously.

PubPeer offers thousands of examples of verifiable claims. Take this comment by an anonymous “Unregistered Submission”, which contains an image the poster created to highlight evidence of manipulation in a paper published in Science. The evidence is all there in the image, which illustrates how some data has been duplicated and presented as being from two different experiments, which is at best a mistake and at worst fraud. Anyone can verify the alleged duplication by consulting the original published paper (as several other commenters did). The identity and motives of the “Unreg” don’t matter: the image speaks for itself.

oodN5OrI do agree with Moriarty that it would be great if we could change the culture of science so that no-one needs to speak anonymously. But I’m not sure how we can go about doing that.

CATEGORIZED UNDER: blogging, ethics, select, Top Posts
ADVERTISEMENT
  • Thomas Browne

    Circles of authors will/do coordinate to protect the status quo and stifle dissent. Those among them who are editors will ensure your papers to go to press. Anonymity is the only way to go.

  • Philip Moriarty

    Thanks for writing this entirely fair, well-balanced, and engaging post, Neuroskeptic.

  • http://nonsignificance.blogspot.com non_sig

    I’ve recently wondered, what would happen if indeed all science were anonymous, but unfortunately I’ve come to convince myself (despite a strong bias in the opposite direction) that it would not be possible. But at least, I think, the possibility (for anonymity) should be given wherever possible.

    “The PubPeer thread on the Blatt editorial is a great example of this [norm] – the vast majority of those commenting are anonymous, yet their comments are innocuous in the context of any type of potential damage to their careers.”

    How does he know? Maybe their colleagues/Professors/employers (I don’t know what the correct word for all at the same time is) don’t like them being at PubPeer? Maybe they (the bosses) fear PubPeer? Maybe they are friends with someone whose paper got criticized there? Or it is a paper of friends of them? Maybe they think every free minute should be spend… p-hacking ;)?

    Credibility is of course another thing and I agree that the credibility of signed statements can be higher (is higher if not data-based and not only a question).

    What would be the negative effects if anonymity would be the norm?

    (Sorry for my bad English.)

  • Pingback: Post-publication peer review: signed or anonymous? | For Better Science()

  • https://www.facebook.com/ScienceCartoons Leonid Schneider

    I have also blogged on this issue, with a “Jacob Hanna” example of how anonymous PPPR can be abused: https://forbetterscience.wordpress.com/2015/12/13/post-publication-peer-review-signed-or-anonymous/

  • disqus_xk55NgnmR2

    Any PPPR comment should live or die on its own merit.

    Yes, comments from highly regarded scientists may initially hold more weight than unclaimed concerns, but one need not be a famous rocket scientist in order to determine that two images are, in fact, repurposed copies and that, no, 1+1≠3. If certain individuals insist on abusing anonymous platforms (e.g. via ad hominem attacks) then these trolls are easily disregarded by any discerning scientist. I have no time to spar with someone who is unwilling to be open or think critically about an issue.

    What’s paramount here is that genuine concerns regarding potentially problematic papers are aired to the public (where authors can directly address data inquiries) instead of filed away to an editor’s inbox and that any patterns of data manipulation(s) by repeat offenders are revealed.

    @Philip Moriarty: Cut and paste newspaper letters? Like some sort of maniacal kidnapper? Couldn’t one just pen a note and not sign their name/ use a pen name or have a close friend pen it for them if they’re concerned about hand writing analysis? Moreover, I could very easily sign any moniker I choose to this post, even if my true name is required. A ban on anonymity won’t prevent me from circumventing the rules. In fact, fake peer review rings thrive on stolen identities. While I agree that sock-puppetry and whatnot is problematic, I can’t believe that it is a direct result of anonymity and a sound basis for eliminating the practice. Anonymity in science existed long before modern PPPR and debates were not always as open and honest as many claim them to be. I wish that discussions could always be open in such a manner, but do not currently see this as a viable option as livelihoods are frequently on the line. Perhaps a time will come when attempting to uphold integrity does not necessarily brand one a ‘troublemaker.’

  • Bill C

    Philip Moriarty provides a link to phyicsfocus which highlights the difference between comments signed by a pseudonym and comments by anonymous “unregistered” posters who may or may not be the same person. Pseudonym consistency (and discoverability of comment history) are tools that can be used in determining the integrity of anonymous comments. For example, anyone here can probably click on my Disqus profile and see that I am a political liberal who likes to read The Atlantic 😉

  • Pingback: Weekend reads: 179 researchers indicted; how to reject a rejection; breaking the law on clinical trial data - Retraction Watch at Retraction Watch()

  • Pingback: 'Making a Murderer' Filmmakers Fire Back at Prosecutor: 'He's Not Entitled to …()

  • Pingback: We are anonymous (or not). We need you to join. We are (mostly) making scientific discussion in the open possible and easy. | Rapha-z-lab()

  • Pingback: Examen de passage | Dernières nouvelles du front()

  • Pingback: Science goes darknet ? – Blockchain for Science()

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Neuroskeptic

No brain. No gain.

About Neuroskeptic

Neuroskeptic is a British neuroscientist who takes a skeptical look at his own field, and beyond. His blog offers a look at the latest developments in neuroscience, psychiatry and psychology through a critical lens.

ADVERTISEMENT

See More

@Neuro_Skeptic on Twitter

ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Collapse bottom bar
+