This instantly makes me suspicious that they are trying to pull a fast one on me. Its also interesting the things that are often not shown. For instance, a qPCR validation of a microarray experiment. Why are they unable to take 10 mins to make a bar graph or excel sheet to be slipped in as supplemental data in the paper? Or just recently, a paper using RNA-seq data. They claim a certain number of genes differentially expressed, but then present only a subset of differentially expressed genes that support their hypothesis. Meanwhile, they also make brief reference to certain classes of genes, but do not show the data………

While I’m on this rant (forgive me) let me add another pet peeve, vagueness in the methodology…in particular vagueness in the informatics. I can not tell you how many sequencing papers I have read where they briefly say “reads were mapped with bowtie/tophat/bwa/etc” and thats it. No mention is made of the parameters used, any of which could alter the data in subtle ways. Unfortunately a lot of bench scientists venturing into informatics simply cut out these details, considering them minor. Even the Bioinformaticians are guilty of it at times. How could I reproduce their results or catch a mistake without knowing the parameters/commands used in the analysis? At least with the statistical genetics papers they actually have equations so you know (if you understand) how they are calculating their results.

Fortunately a lot of journals are now requiring that any new genomics data be deposited in public databases (GEO, SRA, etc) which is a good first step, but its not enough. Typically this means the researchers dump just the raw data, which takes days, even weeks to reanalyze. Of course if you do not know the exact informatics methods used, it can be hard to reproduce exactly (almost impossible if they don’t tell you version numbers of the programs).

The next step in openness will not only be making the data publicly available, but setting accepted guidelines for the reporting of results and methodologies so that it is clear and open to everyone.