I have been guilty of being on the other side of question #2. When the referee asks for an annoying but inessential revision, usually I just ignore it for O(years) and then get back to it when I have the time, patience, inclination, etc. The more you procrastinate, the more you forget the stuff you did in the paper, which adds to the motivation to procrastinate even further.

Perhaps the other reviewer only just sent back his scintillating “It’s perfect, publish as is” review last week, after sitting on the paper for a year.

I recently refereed a paper and when I submitted the report checked a box ‘please don’t send the revised version back to me but to alternative referee enternamehere’. So I was hoping to never see that thing again, but of course it came back two months later. Makes one wonder what that option is good for to begin with.

The most annoying thing about refereeing though I find if I get back rejected papers from other journals, typically without any change whatsoever. I mean, I usually do at least consider that the referee might have raised a valid point before submitting my paper elsewhere. (I usually decline to review in such cases, I feel like the author should get at least a second opinion.)

A heads up: In the September 2008 issue of Discover magazine, stem cell researcher Robert Lanza expounds on the topic of quantum physics. This blog will be expected to comment.

Polite-but-scathing is equivalent to “I see that in Step 2 you have neglected important-well-understood-physical-effect. Unfortunately, without considering this effect, none of the subsequent conclusions are valid.” Most rational scientists would slap their head, say “oh crap!”, and then consider the important-well-understood-physical-effect, rather than choosing to put in a sentence saying “Important-well-understood-physical-effect may also impact our results”, and then return to ignoring it for the rest of the paper.

I would argue that taking this approach should not take a year.

As an author, it is impossible to publish anything even mildly original and against the grain. As a referee, it is impossible to stop something from getting published. Thats peer-review in a nutshell.

I should say this is the situation faced by young people in academia. As I have gotten older, publishing has become significantly easier, but stopping something that I think is crap from getting published is still impossible.

I will be publishing this thread verbatim as part of my paper on the sociology of the peer review system. No edits will be considered.

Regarding point #2, you’ll be glad to note that the ApJ now has a new policy. Below
is an excerpt from the standard response to submissions.

“The Astrophysical Journal has adopted a new policy that manuscript files become
inactive, and are considered to have been withdrawn, six months after the most
recent referee’s report goes to the authors, provided a revised version has not
been received by that time.”

Baby steps….

ST:

As an author, it is impossible to publish anything even mildly original and against the grain.

Huh! The opposite is true as far as publishing in physics journals is concerned. The Referees will usually write a very positive report if you are original. They don’t really check the details in the article. So, if you have some nice original idea, your article will be accepted even if your detailed mathematical arguments are very sloppy

In mathematics journals they also welcome original ideas, but there you can expect a Referee report in which every misplaced comma will be mentioned

I do like (the other) brad’s new policy even though I have a coauthor who would be driven crazy by that rule….

What

“Other typical problem. The person in question sits there and focuses his efforts on one particular puzzle ot solve, and ignores every other constraint from outside his little niche. So when you tell him that subfield x has ruled his result out long ago, they get upset. Even attempts to simply include those relevant results can prove to be difficult.”

Why does that matter? Multiple contradictory hypotheses are part of science. As long as every one knows that subfields X and Y produce mutually incompatible models of a particular system that are incapable of explaining each other’s data, it would be silly to reject papers that try to refine either model, as progress would never be made.

Incidentally, this was the case for the Earth’s mantle (the solid rock portion of the planet between 35 and 2900 km depth, for non geologists here) for much of the late 20th century. Geophysical models could not reproduce the chemical constraints, and geochemical models could not explain the physics.

But if every paper on the topic had been rejected because they didn’t explain everything, then the problem never would have been sorted out.

Dear Sean,

I changed the discussion because your absurd objections proved that the more slow witted readers wouldn’t see the brilliance and transcendant truthiness of my results.

I waited a year to re-submit because I wanted to give you time to adjust to the more intellectual air of Caltech and absorb some wisdom from your desk.
A vain effort, I see.
.
.
.
.
.
.
.
.
.
.
.
.
A joke, of course. I have never written anything, even a parking ticket, likely to end up in Sean’s inbox.

anticipating a critical reading makes authors more careful

But in principle authors do that not to satisfy the referee only, but for their work to be accepted by the community at large, right? Scientific process naturally calls for peer reviews all the time, so a paper comes under real scrutiny after publication anyways, no? What is the real advantage of centralizing the refereeing process, when information is so abundant, and most people don’t get info from books (= traditional journals) any more?

Maybe people really only read abstract and summary these days, so at least one person in the field needs to pay attention…

But in principle authors do that not to satisfy the referee only, but for their work to be accepted by the community at large, right? Scientific process naturally calls for peer reviews all the time, so a paper comes under real scrutiny after publication anyways, no? What is the real advantage of centralizing the refereeing process, when information is so abundant, and most people don

Maybe we could have a version of the arxiv in which people respond to papers with comments

About a couple years ago I remember seeing arXiv implementing a comment feature of some sort, or maybe just a way to send a trackback to an article, I don’t remember exactly. It were there for maybe a few weeks and then disappeared…

I constantly hear people complaining that anything gets published these days (despite that most probably do benefit from such lax standards); papers being rejected due to referee bias is a different problem from low-quality/unworthy papers not being rejected. That implies the traditional peer review process through journal publication is not working the way it should be, no? Yet in the end bad papers won’t survive the scrutiny after becoming public for long, or at least we hope that scientists weed them out, so why put too much emphasis on things getting “published”?

I guess we already do that to some extend by taking into account citations.

But If we look at the wild variation of reviews we receive from TAC/reviewers for mostly similar research proposals submitted in different years, it’s pretty clear that there really are no true standards as if science is in the eyes of beholders. Why put so much emphasis on a single person’s review then?

I work in engineering rather than science, so one difference is that there are more “constructing mathematical ideas and algorithms” that need to be motivated by what one can/should eventually be able to apply them to.

I find the most frustrating thing is papers that aren’t truly padding but also clearly contain just enough that’s new to qualify. The end result is that you get maybe an 8 page paper where 1 page is yet another motivation for a specific task, 1.5 pages is yet another review of how previous people have tackled this/similar problems, 1.5 pages is yet another description of the well known basic mathematical setup (so that the paper is reasonably “self contained”), .5 pages is yet another description of how the experiments were evaluated and why this is appropriate, 1 page is obvious conclusions and yet another listing of references most of which are the ones everyone uses. So 5.5 pages are “duplicating existing stuff”, leaving 2.5 pages of new stuff and experimental validation. Apart from anything else, it takes time and concentration to read those 5.5 pages to be sure they are “yet another …” and there’s not something new hidden within there.

I do wonder if there’s a better way these days to structure scientific/engineering literature these days than requiring every paper to be reasonably self-contained.