If we’re talking about grants, that’s an entirely different situation: the funding body doesn’t have to live with the awardee for the next three to six years, so it can decide based on the science. And in my experience it does. Of course an element of ‘can this person, or these people, really deliver what they are promising?’ enters into the assessment of a grant proposal — but that can be taken into account by the referees. I’ve never been asked to referee a grant proposal from a person or a group that I didn’t know, and my knowledge about what they’ve done in the past feeds in to what I expect them to be able to do in the future.

For fellowships I admit it’s a bit tougher. The fellowship selection panels will not know the candidates, especially for a fellowship over a broad discipline, so they obviously need some input from others. At a junior level they may find they can’t find external referees who know the candidate either. I am told by people who’ve been on those panels in the UK that the external referees, who go basically on science and standing in the community, get much more weight than the referees chosen by the candidates, but it may be different elsewhere, and if you wanted to get rid of the letters of reference from *that* system, I wouldn’t object.

Given that possibility of misuse, I just think that letters of reference should not be essential at all when ranking research proposals and programs for distributing resources specifically made available for them. Having more information doesn’t always help in a good way. People have been struggling in finding proper ways to evaluate others, hiding and unhiding parts of identities like name, gender, sex, race, sexual orientation, etc. for so long in order to fight innate prejudices that having access to more information introduce.

As yet another astronomer’s anecdote suggests, in the end I don’t even think abolishing recommendation for grant/fellowship programs will change anything. It will simply keep the door open to more ideas and people when something goes terribly bad for few unlucky ones. The current system is so inbred that the cost of failure in establishing personal relationships seems just too high.

Your anecdotes are enlightening! But at the level of anecdotes I can also show nearly the opposite, at least at the level of grad school admissions (where this recent discussion started).

I attended Caltech as an undergrad back in the 1980’s, and I can say that we had more than our share of folks who were undeniably brilliant but weren’t necessarily recognized within the academic system. I can discuss one of these cases whose numbers I happened to know. This fellow, quite a bright light and much smarter than me, graduated in physics with (if I recall correctly) a 3.5 in-major GPA and a 960 physics subject GRE. No one had any doubts that he knew his stuff! and he, along with the rest of us, applied to the usual first-tier graduate programs. But it came as a surprise to many — would it surprise you? — that he was rejected nearly everywhere he applied, including Berkeley, Chicago, Princeton and MIT. I don’t know any more details, but it’s only reasonable to infer that his recommendation letters were somehow poisoned and that’s what did him in.

This is not to say that any injustice was done, necessarily; maybe there was a good reason he couldn’t get good letters. But it is a clear case of where a bad letter, while it might not absolutely kill you, can definitely keep you out of the first tier. So I think Mark’s advice is good: one should take great care in whom to ask for recommendations, and be as sure as you can that they’ll make you look good. After all, when you apply to grad school this is pretty much the only degree of freedom you have and so you might as well use it.

1) I served once on a panel to evaluate applicants to a postdoc fellowship which has the unusual feature that it doesn’t take recommendation letters. The application is purely proposal + CV based. On the one hand, those of you who hate the idea of letters could probably use this as slight ammunition for the idea that they could be done away with (though I have no great ideas about how you’d do that in fields where the norm is to be one of many authors on a huge collaborative team). But on the other, I can tell you that a surprising number of the candidates we picked purely on the basis of their research proposal (plus, secondarily, publication record etc) ended up being the folks who also got offered lots of other positions that _do_ take letters into account. What people say in letters, and what shows up in other — perhaps more objective — measures are not orthogonal.

2) I know someone — now a professor at a good research program, much honored by many people — who, while still a grad student, had to testify against his PhD advisor in court. So let’s just say they weren’t on the greatest terms. One bad letter didn’t kill him; it won’t kill you either.

My impression is that a lot of departments have hired faculty in astronomy over the last ten years or so. It’s become more interesting to physicists, so even colleges without a separate astronomy degree now have an astronomer or two lurking the halls. Moreover, I think with the wealth of publicly released data from large surveys and space-based telescopes, it’s much easier for faculty to have interesting research programs even at a smaller college. Back in the day, if you didn’t own a 4m piece of glass somewhere, you were pretty much screwed trying to run a cutting edge program through only allocations from the national observatories. Now, you can do hot stuff no matter where you are. This has greatly expanded the research opportunities for students, beyond just what’s available through REU’s (which I think have expanded as well, but I’m less sure).

has the # of reu’s exploded that much in the past couple years in astronomy/physics?