One thing I have been thinking about in how we should make progress from this point onward is to indeed put some of the responsibility in the hands of the journals. Why not demand from top journals that they do two things with the very high subscription fees we have to pay them:
- provide a mandatory data repository service for all data published in the journal.
- give grants to junior researchers or prizes to master students that perform replication studies, written down in white papers that are also linked to the digital record of the published papers.

From academia, we could then follow up on this. If replication study papers would be archived in an online repository linked to the original article, this would have two benefits:
1. We would be more informed on the true value of a published study
2. We could decide to have these replication papers count to some degree in in the junior researchers’ track records, so that it is actually good (and valued) training to start with replicating prior studies.

I like the idea of pushing more raw data public, but the issues surrounding useful data curation & storage and volunteer privacy are non-trivial in many fields.

Wikipedia is helpful: http://en.wikipedia.org/wiki/Order_statistic

It presents the joint distribution of order statistics but does not show the moments, which would be nice for understanding this fraud detection algorithm.

I wanted to comment on the whistleblower aspect of the story, I think it’s very important. First, I think we place a large burden on people like Uri, which may be inevitable to some degree, but still not really fair and we all owe him our gratitude for taking something like this on.

More importantly still, as you note, there has to be a lot of concern about not getting one’s accusations wrong, especially publicly. The steps that Uri takes to minimize that possibility are very important and should be applauded. Contacting the researcher before publication is obviously important, as is the fact that he contacted the relevant authorities instead of pursuing justice on his own. Also, the attempt to replicate his findings with other papers by the same author makes false positive accusations far less likely.

It necessarily also means that there will be more cases that will be missed, or dismissed as too ambiguous to pursue. I think that’s a trade off we have to make at one point or another, I’m glad that we’re starting at a point that’s reasonably conservative.

I’d also like to support Uri’s call that journals (or authors) publish data. There are certainly exceptions where this is a little trickier, but now that we can post data on the internet, there are many fewer excuses for not being transparent.