The New York Times has an article that reviews the problems with peer review. I don’t know what to think, as it has something of the “and the other side says” air to it, never really coming to any conclusion. There’s a lot of shoddy crap being published in stuff like the Tuvan Journal of Entomology, and always has been, but the big issue is when crap gets into Science and Nature. Ultimately science is a social enterprise based on trust and long term self-correction. There’s a lot of noise in the system, but I don’t see any other alternative out there.
One issue that I’ve been wondering about is that in many of the biological and social sciences many new mathematical and computational techniques are being pushed really quickly, and I’m not totally sure if there’s enough comfort with these methods for the scientists to utilize them without error and the peers who check up on them to catch those errors. Sometimes it can be pretty embarrassing. A precursor to what I’m alluding to was the whole Mitochondrial Eve fiasco, where the original team wasn’t totally comfortable with the new phylogenetic techniques and made some serious methodological errors. A phylogeneticist even told me that in the original paper there was a mistake where the length of a particular branch of the tree was conflated with genetic distance when in reality the tree only reflected the bifurcation into clades, the “long branches” being an artifact of presentation by the program.
Update: Brown gaucho has a lot more commentary.
Update II: Janet has more.