The Truth, Respectfully

By Sean Carroll | April 11, 2008 11:51 am

So you’re listening to a talk, and the speaker introduces a crucial step which you know — or perhaps only suspect — to be completely incorrect. What do you do? Do you raise your hand and point out the mistake? Or file it away momentarily, planning to ask them about it in private afterwards?

And does your answer change if the speaker is a senior scientist who will some day be writing you a letter of recommendation? What if it’s a fellow graduate student giving their first-ever technical seminar, and you know them to be intimidated by all these smart people in the room?

A Lady Scientist and PhysioProf have been talking about these issues. The former wonders whether there shouldn’t be some solidarity among grad students not to make each other look bad during journal club presentations, while the latter says that good perceptive critical questions are always in order.

My own attitude is pretty straightforward, and close to PP’s: it’s never impolite or out of order to ask appropriately probing questions about the material being presented at a scientific talk, regardless of the status of the speaker or the audience. It’s science, and we’re all on the same side; it doesn’t do anyone any favors to hide the truth in order to save someone’s feelings. Science is bigger than any of us, no matter how young and inexperienced or old and respected (feared) we may be. Not only should listeners feel free to ask any reasonable question of the speaker, but speakers should be honest enough to admit when they have said something that might be incorrect, rather than twisting around to find justifications for a slip-up. We’ve all made them; or at least I have.

To the extent that there is any sort of competition going on, it should not be “speaker vs. audience,” but rather “all of us vs. the natural world.” However, having staked out that absolutist position, it’s extremely important to recognize that we live in the real world. For one thing, many audience members tend to blur the distinction between “asking a good question” and “being an asshole.” There are people out there, one must admit, who tend to view seminar questions as a venue for them to demonstrate how smart they are, rather than learning about the subject matter in an open and collegial environment. There’s no excuse for that, and the guilty parties deserve to be smacked around, if only symbolically. Still, it’s no reason for the rest of us to equate hard questions with egotistical puffery, nor to soft-pedal questions that really are sincere. The biggest benefit of a talk, from the viewpoint of the speaker, would be to actually learn something from the questions and comments offered by the audience.

The other complication is that there is a competition going on, whether we like it or not. I personally don’t like it, and would vastly prefer to live in a utopia of unlimited resources where such competitions were unnecessary. But in the real world, there is a limited collection of goods — jobs especially, but other rewards of the profession — and a large number of people competing for them. And that competition never turns off. Academics are always judging each other, inevitably, and will use those judgments when it comes time to recommend or hire or give prizes to each other. So a real seminar is not simply a value-neutral examination of the facts; it’s a social milieu, in which interactions have real consequences.

Which is not to say that we should ever shy away from asking hard questions. But there are different ways to ask hard questions, and there’s nothing wrong with choosing the tone in which such questions are asked to match the occasion. Graduate students giving their first seminars need to learn that they will get asked tough questions, and that it’s okay — it’s not a devastating critique of their worthiness as scientists, it’s simply part of the process to which we are all ultimately subject.

A common technique to help students ease into the responsibility of giving talks is to have students-only seminars where the faculty are not permitted. The motivation for such things is admirable, but ultimately I don’t think they are a good idea. (As a disgruntled senior colleague once said, “Sometimes I learn something from listening to the students.”) Breaking down the barriers between “faculty” and “students,” and beginning to think of everyone as “researchers” and “colleagues,” should be an important goal of graduate school. It can all be intimidating at first, but it’s ultimately beneficial to learn to treat these artificial hierarchies as administrative annoyances, not natural categories.

The most successful graduate students are the ones that start thinking of themselves as colleagues right away. Go to the seminars, sit in the front, ask good questions, participate in the informal discussions afterward. It’s a big universe out there, and we’re all struggling to understand it, and working together is our only hope.

CATEGORIZED UNDER: Academia
ADVERTISEMENT
  • http://www.cita.utoronto.ca/~ljdursi/ Jonathan Dursi

    It’s interesting to me how much of a cultural thing this is, not just within disciplines (although certainly there) but even department to department. Where I did grad school, people were very reserved in asking questions, and would certainly wait to the end of the talk regardless; in my current gig at a research institute, everyone is cheerfully merciless and if you get through a talk without being interrupted (and usually challenged) at least 5 times, it means no one was very interested in what you were saying.

  • Moshe

    I think there is a difference between asking for clarification, with the underlying assumption that there could be one, and discovering a fatal flaw (especially if it is a well-known fatal flaw). The former contributes to the conversation being productive, the latter does not. In my experience then most people would not bother pointing out the elephant in the room , one of the reasons why complete silence during a seminar is a bad sign…

  • jackd

    The closest thing to a universal rule is one you implied above: Don’t be an asshole. You are not the local version of Gregory House, M.D. whose utter brilliance will allow you to behave like a jerk. Do Unto Others, etc.

  • http://blogs.discovermagazine.com/cosmicvariance/sean/ Sean

    Moshe, I know what you mean, but — if you were the speaker, would you prefer that everyone sat silently through your talk, knowing you were full of crap but reluctant to say so out loud? It’s not an easy question, that’s for sure.

  • http://tyrannogenius.blogspot.com Neil B.

    Sean, I think you are saying that instead of an overt statement “You are wrong about _______”, you mean that a good option is to ask a smart Socratic question that will lead the speaker to realize the error, and then perhaps find a way to wiggle out of it gracefully? That seems to me the best way to go. It is right to care about embarrassing speakers, but remember that the audience is being abused somewhat by any mistake that isn’t corrected – that should matter.

  • http://www.farrellmedia.com John Farrell

    Sean, I may have told you this, but when he was in the army during WWI, Lemaitre got cited for insubordination for pointing out to his senior officer/lecturer that there was an error in the artillery ballistics manual. His temerity cost him any chance of officer grade, and while he joked about it in old age, it was clearly one of those life-learning experiences that rankled.

  • String Theorist

    Excellent post, Sean!!!!

    I went to grad school at a place where there were many bigshots, but there was the concept of “faculty lunch” which the grad students were kept out of. I did one postdoc at a place where there were only semi-bigshots, and the lunch was open for all.

    At which place do you think I produced more papers per year?

    More importantly, which place do you think produced more papers per year?

  • SLC

    I think the personality of the speaker has to be taken into account. When I was a graduate student, I sat in on a presentation by the late Nobel Prize winning physicist Julian Schwinger. My thesis adviser, who was also present, had the temerity to question something Prof. Schwinger said and got his head handed to him.

  • Moshe

    The context I had in mind is slightly different, where there are some well-known problems, certainly well-known to the speaker. Most recent example is a seminar I attended where the speaker derived a probability distribution which was non-normalizable and strongly cut-off dependent (aka completely arbitrary), both of the problems are so well-known that you probably could guess exactly the topic and perhaps even the speaker…This is an example where asking a question is not likely to generate new information.

    But, I agree completely with the general sentiment, and everything you wrote above, just that your first sentence is a bit more specific than the rest of the post, which threw me off a little…

  • M

    Not telling someone when you disagree is to not take them seriously, it is supposing that they aren’t capable of discussion and reasoned thought. Part of treating graduate students as scientists is presuming that we are worth engaging with. Of course, there may be people who delight in humiliating the speaker, but that’s a different story.

    but Sean, how often does this happen? is it really common for a major point (rather than an aside, or a qualitative explanation, etc…) is totally flawed?

  • anonymous

    There’s a good Feynman story about this:

    “I sometimes attended graduate physics seminars as a freshman to see if I might hopefully be able to absorb anything. Usually they were mostly over my head. I remember attending one seminar at which Feynman was also in attendance, sitting toward the front of a large lecture hall surrounded by a few colleagues. The lights were dimmed, and the seminar was presented by a graduate student from a prestigious East-coast university that shall go unnamed. As I recall, this was a presentation of his doctoral thesis work which he in the process of completing. I didn’t pay too much attention to all of the overheads, as the math was beyond me, and I noticed that Feynman’s head was bobbing and he seemed to be dozing. After about half an hour of rather tedious equations with no audience participation, Feynman’s head suddenly jerked up and he said loudly, “You can’t do that!” The confused grad student froze in confusion, then asked what it was that was wrong. Feynman made a brief reply, and then some of his colleagues, apparently in the belief that he hadn’t been paying attention and was confused, tried to defend the student. Feynman then jumped up in disgust and went to the board to scribble up a few equations. There was a somewhat stunned silence. From what I was able to gather, the student had somehow mixed equations related to wave and particle theories of light in a way that they couldn’t be used together. I don’t know if he was able to salvage any of this thesis work – but it pretty much put an end to the presentation. I felt a little sorry for the grad student!”

    (taken from this website: http://www.apnoia.org/ivantic/fizika/feynman/index.html )

  • http://blogs.discovermagazine.com/cosmicvariance/sean/ Sean

    Moshe, yes, I certainly agree with you in that context — if the speaker is saying something “incorrect” that they are already aware of, and are willing to look past the purported incorrectness, there is little point in raising a stink about it.

    M, it is certainly not very common — for a faculty-level speaker, I think I’ve only seen it happen once or twice. But for grad students, especially someone who is giving a talk at a journal club (and therefore about someone else’s work), it’s understandably more common; they haven’t been doing this stuff long enough to be familiar with all of the major kinds of mistakes. But making mistakes is part of the process; you can’t learn to juggle flaming torches if you’re unwilling to risk getting burned.

  • The ALmighty Bob

    Don’t know about you, anonymous, but I’d prefer to find out then rather than when defending my thesis…. At least that way it might be salvageable.

  • R Senic

    “Gotchas” are something we do when we are being geeks.

    The problem with “gotcha” is that in highly technical fields, responding accurately to one is hard to do live and on the spot, and neither side is likely to be able to demonstrate much rigour. This is why we circulate and review formal papers, *especially* on topics at the bleeding edge of existing theory, and geek out over coffee or AIM/MSN/Jabber.

    Informal discussions in various media and settings are great for brainstorming, for hashing out partially-formed ideas, and for “gotchas” which are useful in writing up a paper in the first place (if only to add some wording that stems similar “gotchas”). Formal talks are less great for this, but often a better way of catching up with other people’s work with which you might not be intimately familiar.

    In highly technical fields, it’s pretty hard to keep up with what Dr Jones is doing if you are busy with your own research, and when you make a few hours to listen to formal talks, the last thing you want is to have the talk derailed by a series of someone else’s “gotchas” which might be better answered in detail in writing, rather than in the margins of presentation foils (wow, sign of age there) or whiteboards.

    Sure, it’s not much fun to find out you’ve given an hour long talk on something that is fundamentally wrong, but it’s alot worse to have a talk which is *reasonable* (even if partly wrong) be totally derailed by someone who is unprepared to fully justify the derailment (or make it up to the rest of the audience).

    Faculty who make a habit of derailing students’ presentations — no matter how informal — are not helping a student learn how to deliver useful arguments or respond to criticism in a well-reasoned, rigourous fashion. Even in the Feynman anecdote above, there is an unpleasant smell of argument from authority.

    I think it would have been more useful to have found the student afterward, gone through the apparent error, and helped walk the student through the options for clarifying the error after it escaped into the wild during the talk.

    On the other hand, being exposed to bullies, unreasoned criticisms and out-of-the-blue demolitions of fundamental assumptions is also good preparation for “the real world”.

    On the other other hand one can take this too far. By way of answering the unsourced Feynman anecdote, an “anecdote” at the link below is, sadly, an only mostly ridiculous exaggeration of the experience of a derailed formal talk.

    Search for “1982 Symposium” in the page at the link. The name “Felton” is probably a joke name with no conflicts at the time of writing (1987) rather than a prescient reference to a very young Edward Felten, whose name seems to be misspelled frequently:

    http://www.ccs.neu.edu/home/shivers/autoweapons.html

    Then again, perhaps Olin Shivers remembered the future?!

  • R Senic

    Er… “unsourced” in my comment above was unfair and inaccurate. I simply did not see the “taken from this website” link at the bottom of Anonymous’s comment. My apologies.

  • http://deleted Simon DeDeo

    More than 90% of the time I’ve been in such a situation (as an audience member), I’ve been wrong. Not necessarily wrong-wrong; often times the speaker has made an assumption that only later she’ll reveal. Perhaps I’ve only gone to very good talks; very rarely have I seen a situation in which a complete talk falls apart, and never has it been unambiguous.

    Especially when going to a “near-neighbour” talk — i.e., one out of my subsubfield — I’ve learned to keep quiet when I see the FATAL FLAW OMG and bring it up only after a number of slides have gone by (or after the talk), and only in a very much “I think I am missing something…” kind of way.

    In terms of doing it with beginning graduate students; often times they are the least aware of the background assumptions of the field, and I’ve seen them fall apart when questioned on it. I think there should be some solidarity going on.

  • http://deleted Simon DeDeo

    …looking over the posts that you point us to, it seems there are two issues.

    One is “at a journal club, do you hammer people?” Answer is yes — it’s an informal situation, you won’t derail anything, everyone’s on the same team, the work is embryonic, and you have the right to suddenly turn ten minutes of the club into an “educate me” thing, which is the “worst case” scenario.

    The other is “at a formal talk, with slides and stuff, do you hammer people?” Answer is no — you are most likely wrong (75% of the talk will be peer-reviewed by those closer to the question that you) and you will derail a talk that other people want to hear and the presenter needs to give.

  • Annie

    Sean, could you elaborate on your opposition to students-only talks? In my department, we have scads of talks that graduate students participate in — three different journal clubs that run weekly during the year and are really propelled forward by grad students and postdocs, plus special events/exams and the like, and plenty of outreach opportunities for public talks.

    Our journal clubs are pretty ideal environments, I should say. The question from A Lady Scientist was actually pretty shocking to me, because I can’t imagine a journal club environment where it wasn’t TOTALLY clear that the weird, unexplainable, or just plain wrong stuff from the paper wasn’t the responsibility of the presenter! And there’s always a lot of conversation — it’s totally OK to say, as a presenter, that you don’t know the answer to the question, but if somebody else does know, they’re not going to just sit there silently since it’s “your week” — they’re going to share.

    But we also have, during journal-club-offseason, a set of talks for grad students and postdocs. Partly this is to ease students, generally, into “giving talks in a friendly environment,” but I also think we’re pretty good about self-policing and being responsible rather than using the ‘easy’ audience as an excuse. I think most of us do a good job of using it as a *special* kind of opportunity. And, of course, it’s not always about giving “talks,” but a PARTICULAR talk — job talks, exam talks, important conference talks, potentially life-changing talks. Additionally, I don’t think this series decreases the number of talks students give elsewhere in the department, or decreases the amount of responsibility overall; it’s just an ‘extra’ commitment that we all make and all benefit from.

    Our grads-and-postdocs only event also does a great job of helping to create and support a feeling of community. I have actually noticed that since we started having more events for grads & postdocs only, I have become much more forward about asking questions — not just at these events, but in general.

    So, I think like most things in grad school, it’s a tool and an opportunity, and of course it can be misused, but I think it also serves (maybe “can serve”?) some specific purposes.

  • Costanza

    One thing that often helps, if we’re talking grad student, experienced (feared) scientist, academia, private sector, whatever…is the rapidly disappearing art of civility and manners. Altogether too many take the attitude of, “I’m a (***you enter here)*** so that gives me carte blanche to be an asshole.” To paraphrase Winston Churchill, “If you must kill a man, it costs you nothing to be polite.”

  • http://blogs.discovermagazine.com/cosmicvariance/sean/ Sean

    Annie, my objection is not very strong; more precisely, there are strong arguments, but on both sides, and I’m sympathetic. Having a place where grad students can make mistakes away from the totalizing gaze of the faculty definitely has its benefits. But undermining the student/faculty distinction, and beginning to treat everyone as equals, is also a laudable goal. I tend to feel that the latter wins out over the former, but certainly understand those who feel otherwise.

  • Annie

    Hmm, maybe we could amend the rules: only grad students, postdocs, and faculty members who actually LIKE the idea of treating young’uns as equals ;).

  • http://www.cita.utoronto.ca/~ljdursi/ Jonathan Dursi

    A grad-student only talk series not only helps the students learn to be comfortable giving talks, but *also* helps the students learn to be good audience members in a group of their peers, including important skills like asking hard questions without being a jerk, asking questions when one is confused and has a good idea that it’s not their fault, and being polite because the tables will be turned fairly soon. Just as its easy to be intimidated giving a talk to senior faculty in your first couple years of grad school, it’s easy to be intimidated into not asking questions.

    Both of these goals (and roles) are important, and I like grad student only sessions for exactly this purpose. It also helps remove the over-inflated sense of `grad student solidarity’ mentioned above in talks, as when it’s all grad students there’s no `them’ to embarrass one of `us’ in front of.

  • Pingback: HowTo: ask questions in a seminar « Entertaining Research()

  • Pingback: The Truth, Respectfully : Anand’s Scrapbook()

  • http://lablemminglounge.blogspot.com/ Lab Lemming

    So do you interrupt the talk, or wait until the end and then ask?

  • http://www.dorianallworthy.com daisy rose

    Dark Matter a movie reviewed in the NYT just – yesterday.

    It is the subject of Novels and poetry and art and always in nature – the tried and true Challenged – there is always a fight ! the Vanguard appears.

  • http://backreaction.blogspot.com/ B

    Hi Sean, very nice post! Regarding the well-known problems Moshe mentioned above, I’d think it’s a point for ‘more a comment than a question’. Even if the problem is well-known, esp. if there are students in the room you can never be sure everybody has heard it before. But I’d agree that starting a longer discussion on always-the-same question in the beginning of a talk can be extremely annoying if you’re among those who actually wanted to hear the speaker and not the audience. Best,
    B.

  • Dany

    Moshe:”The context I had in mind is slightly different, where there are some well-known problems, certainly well-known to the speaker. Most recent example is a seminar I attended where the speaker derived a probability distribution which was non-normalizable and strongly cut-off dependent (aka completely arbitrary), both of the problems are so well-known that you probably could guess exactly the topic and perhaps even the speaker…This is an example where asking a question is not likely to generate new information.”

    But remember W. Pauli vs. C.N.Yang at IAS presentation. Was Pauli wrong or behaved wrong?

    Regards, Dany.

  • Haelfix

    A good rule of thumb imo, is to think about it for at least 5 minutes before you ask the question. Try to make the question brief and tailored so it gets a brief yes/no reply so as not to derail the lecture on a tangent. If he/she is well passed that point, save the question for after the lecture.

    Otoh if the lecturer doesn’t know what he’s talking about (all too often) or the subject is so grotesquely speculative, silence should remain golden. Its just not worth making a fool of the person or of yourself (eg everyone else knows how speculative or wrong the idea is).

    The setting is important too. For instance, if you are listening to QCosmology speakers, where the entire subject is by its nature 3 or 4 steps into theory never never land, pointing that very fact out accomplishes nothing and just irratates.

  • Dave

    It seems to me that it is a bit irresponsible to sit silently in the audience and allow a speaker to present a result that relies upon some clear error. After all, there are likely to be many in the audience that will be misled if you don’t point out the mistake. Of course, it is good to be polite and to try to minimize the humiliation of the speaker if you can. But, anyone giving a scientific talk should expect the audience to ask difficult questions. Also, scientists really need to be able to accept their own errors and move on, so I don’t think that we are doing students much of a favor by allowing them to avoid confronting their mistakes.

    I have known a couple scientists who have great difficulty in accepting their mistakes, and it really seems to me that this does much more harm to their scientific productivity than the actual mistakes do.

  • Jimbo

    Grad student seminars are wonderful because elimination of the `head trip’/fear-factor of faculty, gives the student a`dress-rehearsal’ into the dynamics of presenting a technical talk, learning time-management & visual aids skills, & gaining confidence of delivery, without fear of being intellectually gutted.
    Research seminars are no-holds-barred, with pirhanas and barracuda’s always present. Neophytes should always vet their talk the day before to a knowledgeable colleague to minimize hemmoraghing the water. When Great White sharks are present, like Feynman or Pauli, follow the advice given to Weisskopf by Peirels:
    “Knock on the door of the Great White the morning of an afternoon talk, and tell them precisely what you intend to say. They will bitch & moan, about this & that point, but no matter; give your talk exactly as you had planned to. Great Whites will remain mum, because they already told you so, and hate to repeat themselves !”

  • http://orgprepdaily.wordpress.com milkshake

    I think Feynman was a complete dick in this respect, being unpatent with inept speakers at seminars – and he should have known better for he has been on the wrong end of stick in his younger years, from Oppenheimer and others.

    I work at a chemistry+biology institute and we get applicants for staff job/postdoc positions giving their research presenations all the time. Sometimes they are not very experienced presenters or they may have a language problem so sitting through the talk can be a mild duress – but the speaker suffers more, so it is OK.

    I think it is a matter of common decency to be polite and if there is an inept part in their talk it is best to argue about it discreetly, one-on-one. They have to survive the whole interview day and if you rip them apart publically right at the beginnig, you are doing no favor to yourself or to them. Its bad enough they had unimpressive job talk, you don’t need to rub it into them in withering questions in front of the whole department.(I feel less charitable to pompous and famous people in this regard). It helps to remind to ourself what it was like interviewing for a job.

  • Mike

    I agree with Sean that a good question/comment should always be asked/given.

    One thing that makes me proud to be a physicist is that even the leaders of the field occasionally say “I don’t know” or “I don’t understand.” There is an understanding that no one knows everything, and that lack of understanding is about as likely to be caused by distracted attention, limited background, poor presentation, or incorrect conclusions. Of course, when a leader of the field says “I don’t understand,” there is a much stronger insinuation that the lack of understanding is due to the ideas being incoherent. But I think this is because, in the end, when a leader of field doesn’t understand something, this is the reason why.

    My graduate advisor once found it amusing to speculate how much of the high-energy physics literature would survive if it were purged every time a paper was learned to be in some way “wrong.” Some of the most prized work in physics would now be seen to contain comments or arguments that are misguided or naive. The important thing is to be “wrong” in the “right” way: good work builds upon solid intuition from analogy and is otherwise insightful enough to survive aspects of speculative details. If a person’s work is routinely “wrong” in the “wrong” ways, then it is ultimately good for science that this person gets pushed aside in the field. Of course this is very sad for the person involved, but this is just the sad fact of life that not everyone can do anything he/she wants.

  • http://www.savory.de/blog.htm Stu Savory

    On a similar topic, Sean, what about errors in textbooks?

    Back on the 10th of October 2006, I took Jim Kakalios to task in my blog for a simple mathematical error in chapter one of his popular science book “The Physics of Superheroes”. An error on which he bases his argument for all subsequent chapters. He acknowledged the error in an email and agreed to put it in his (online) errata sheet.
    See http://www.savory.de/blog_oct_06.htm#20061029.

    But never did so.

    Now what does one do when the textbook is wrong and the author refuses to correct the error? Name-and-Shame doesn’t appear to be working…

  • Pingback: Savage Minds: Notes and Queries in Anthropology — A Group Blog » Around the Web()

  • http://math.ucr.edu/home/baez/ John Baez

    Sean wrote:

    Moshe, yes, I certainly agree with you in that context — if the speaker is saying something “incorrect” that they are already aware of, and are willing to look past the purported incorrectness, there is little point in raising a stink about it.

    There’s also the poor audience to consider. I don’t think “raising a stink” is good, but if there are people in the audience who might not know the speaker may be doing something wrong, they deserve to be told. Quickly, efficiently, and gently.

  • Moshe

    John and Bee, I’d agree with that, and in my home institution, where I am responsible for the education of our students, I am likely to make a short comment about well-known pitfalls, especially if the speaker doesn’t. I am just saying this is something to be aware of, silence does not mean complete agreement with everything that is going on, and if you discover some really obvious flaw, it may just be part of the conventions of the field – something people are aware of and are willing to ignore at the moment.

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Cosmic Variance

Random samplings from a universe of ideas.

About Sean Carroll

Sean Carroll is a Senior Research Associate in the Department of Physics at the California Institute of Technology. His research interests include theoretical aspects of cosmology, field theory, and gravitation. His most recent book is The Particle at the End of the Universe, about the Large Hadron Collider and the search for the Higgs boson. Here are some of his favorite blog posts, home page, and email: carroll [at] cosmicvariance.com .

ADVERTISEMENT

See More

ADVERTISEMENT
Collapse bottom bar
+