I was warned off social psychology years ago by a friend (who was a research psychologist) because of the field’s propensity for ‘sexy’ results which get a lot of media play. As a lay person he doubted I could tell the fake from the reliable, the one off from the replicable. Later someone else told me about how dispiriting it was engaging in data dredging to find something publishable in a lab where they were working, even though many of the principals involved obviously assumed that the results weren’t robust. Keep that in mind when you read about the mind-boggling scientific fraud perpetrated by Diederik Stapel. It looks like he might be the Bernie Madoff of psychology. The Lehman Brothers to the Long Term Capital Management of Marc Hauser. Here’s a taste:
Stapel’s work encompassed a broad range of attention-catching topics, including the influence of power on moral thinking and the reaction of psychologists to a plagiarism scandal. The committee, which interviewed dozens of Stapel’s former students, postdoctoral researchers, co-authors, and colleagues, found that Stapel alone was responsible for the fraud. The panel reported that he would discuss in detail experimental designs, including drafting questionnaires, and would then claim to conduct the experiments at high schools and universities with which he had special arrangements. The experiments, however, never took place, the universities concluded. Stapel made up the data sets, which he then gave the student or collaborator for analysis, investigators allege. In other instances, the report says, he told colleagues that he had an old data set lying around that he hadn’t yet had a chance to analyze. When Stapel did conduct actual experiments, the committee found evidence that he manipulated the results.
Here’s the controversial thing I’m going to say: instances of flamboyant fraud are probably far less of an issue in a discipline like social psychology than more subtle biases and systematic incentives. Similarly, someone like Bernie Madoff was notable for the havoc he caused on an individual level, but he paled in the aggregate against the death by a thousand cuts that occurred as the housing market collapsed under the pall of modest “liar loans” and the like. And this isn’t just a problem with science. ‘Sexy’ social psychological research is routinely blasted by the press, and devoured by the public. Incentives matter, and Diederik Stapel is just the reductio ad absurdum of scientists who respond to the repeated 15 minutes of fame which the press provides. Most of the time it’s not so blatant, nor frankly so malicious in consequence. I doubt that there are many Diederik Stapel’s in the field, else he wouldn’t have gotten so far with bluster and bluff.
Razib Khan’s degrees are in biochemistry and biology. He has blogged about genetics since 2002, previously worked in software development, is an Unz Foundation Junior Fellow and lives in the western US. He loves habaneros.
November 1st, 2011 at 12:38 am
There are more Stapels than one would like to think.
I would contend that there is at least one narcissist within social psychology who is a “darling of the media” in virtually every country. But perhaps they don’t all indulge in just inventing data as Stapel did.
Aric Sigman in the UK is a case in point. “How using Facebook could increase your risk of cancer” – and that passes as the “science” of social psychology!
http://www.guardian.co.uk/commentisfree/2011/sep/23/bad-science-ben-goldacre
November 1st, 2011 at 2:32 am
I’m completely horrified by this. I was horrified by Marc Hauser, but this is worse. Maybe because it feels more personal in that he visited IU and gave a talk when I was there, (Rich Shiffrin – cognitive memory guy – was full of disbelief but not in the you are faking data way), and my adviser, a recent graduate, and me tried to replicate/extend one of his findings (and failing – he claimed that the Ebbinghaus illusion was influenced by social status, or something like that).
I want to defend social psychology somewhat in that I do think it, over time, finds out interesting things about humans and human social behavior. But, it isn’t easy, it is messy, it is filled with little mini-theories, and plenty of fashions (but all of psychology is, I think).
And, I would think that this is an area that would particularly suffer from file-drawer issues.
Sometimes I despair.
Find an interesting person to sequence, so I can feel I contribute again.
November 1st, 2011 at 2:42 am
#2, just to be clear re: social psychology as a discipline, my primary point is that there needs to be a ‘buyer beware’ attitude among those outside of the discipline because of the media friendly tendencies of the results. inside the discipline the problem toward confirmation bias is going to be the same as other statistical sciences.
November 1st, 2011 at 4:11 am
I majored in behavioral psychology and it’s a scary scary cult. I didnt like it at all. If I asked my teacher about actual neuroscience she’d just ignore me. I know it’s not the same as social psy but I really wish people could know how bad it is. Someone needs to do an expose on it so we don’t have so many Psy majors like Fareed Zakaria pointed out.
November 1st, 2011 at 8:08 pm
150 papers that Stapel has published since 2004
This is pretty much all one needed to know. It can’t be good science if it is two papers every month.
November 2nd, 2011 at 9:30 am
As a card carrying-social psychologist I am somewhat disturbed by the fact that Razib Khan takes this opportunity to blast an entire discipline. The article is brimming with speculation, insinuation when one has suspect (e.g., because he was “warrned off” social psychology) that he doesn’t really know a whole lot about it. To be sure, what Stapel did is inexcusable. As far as any science is concerned, he committed high treason, and he should be dealt with accordingly. As a social psychologist finding this out about Stapel is like discovering that your sibling is a serial killer.
At the same time, I do commend the author on comparing this to Bernie Madoff, Lehman Brothers etc. In Stapel’s case, as in these other cases, once some of the facts come out, you have to wonder why chores of people working with him did not suspect anything because it is so blatantly obvious that what occurred was “too good to be true.” Most likely, we have to look at how people’s expectations guided what they saw–or what they wanted to see. We have to look at blind spots and distortions (many of them self-serving) that made students, bosses and even prominent collaborators of Stapel’s miss what seems obvious when viewed from a distance. Ironically, this is precisely the sort of thing that social psychologists study–and most of them do that in a very unspectacular way that hardly ever reaches the front pages (even when it arguably should).
November 2nd, 2011 at 3:09 pm
What always scares me when I hear stories like this is how many scientific frauds haven’t been caught. Science is self-correcting over time, but fraudulent papers can set fields back years or even decades if they go undiscovered for long periods.
November 3rd, 2011 at 12:08 am
But what is fraud actually? See Simmons, Nelson & Simonsohn in Psych. Science: http://pss.sagepub.com/content/early/2011/10/17/0956797611417632
The flexibility in data collection and analysis that SN&S describe is pervasive in social psychology, and therefore a lot more worrying than exceptions like Stapel (from the perspective of scientific integrity).
November 3rd, 2011 at 5:23 am
“Social psychology” in the absence of a good understanding how the brain works is like alchemy without physics. Unfortunately the Madoff/Goldman-Sachs analogy is a good one: Stapel committed outright fraud but the main problem is the horde of psychologists that publishes real, but uninterpretable, data. If you are interested in the mind, study the brain.
November 3rd, 2011 at 8:23 am
What is social psychology? It’s been around for a while. It’s produced almost nothing useful. It certainly hasn’t justified its costs. Why does it still exist?
Accepting its own self-definition, social psychology is a way of studying social and psychological phenomena using the social science methods. The sine qua non of modern social psychology is its adherence to modern social science methods, like hypothesis testing, statistical methods and peer reviewed journals.
When it comes to studying social and psychological phenomena, the formal field of ‘social psychology’ certainly has no monopoly. There are many other ways of studying and talking about people, including but not limited to other academic fields, journalism, philosophy, religion and chatting on the internet.
Now that ‘social psychology’ has been around for a while it is a fair question to ask whether the methods of social science have caused ‘social psychology’ to shine more brightly than any of its rivals in the firmament of ways of studying and commenting on social and psychological phenomena.
I don’t think so.
Social psychologists are held in no great esteem by the public. They certainly are held in no greater esteem than any of their rival studiers and commentator on social phenomena. When anyone looks over the vast expanse of human behavior there is almost no area to which the public turns to social psychologists as the primary, most interesting, most thought-provoking experts. As far as the public is concerned, ‘social psychology’ is a dud. It promised a mountain and has delivered a mouse.
The only concept I can think of which has escaped from the den of social psychologists into the common discourse is “cognitive dissonance,’ a more than fifty year old concept associated with the name Leon Festinger. The words ‘cognitive dissonance’ are used occasionally even today in the 21st century. Except they are used in ways which have almost nothing at all to do with the hypothesis put forward by Festinger and his colleagues. The term survives, but their meaning has changed.
Other than that, has anything done by social psychologists as social psychologists had any great influence? And if it has had influence, has it survived any tests outside university laboratories?
I don’t think so.
The question of whether or not social psychologists cheat is unimportant. Neither social psychology done by honest social science researcher nor social psychology done by cheaters has amounted to a hill of beans.
The more important question is whether the use of social science techniques like hypothesis testing, statistical methods and peer reviewed journal have justified their costs. If no one listens and cares, why bother anymore? If someone with no training in social science techniques can put forth an idea about social and psychological activity which gets attention and comment and thought and appreciation, what is the added value of social psychology?
PS I have a PhD in the field, have taught it at the undergrad and grad levels and a few publications in peer reviewed journals (under the name I used before I changed it, so don’t bother googling me for my old stuff).
November 3rd, 2011 at 3:00 pm
“Instances of flamboyant fraud are probably far less of an issue in a discipline like social psychology than more subtle biases and systematic incentives.”
I don’t get it. Are you saying that, say, biochemistry has a higher proportion of outright fraud to subtle bias?
Since I don’t know the answer I’m just going to leave this link here: http://retractionwatch.wordpress.com/category/by-subject/basic-life-sciences-retractions/
November 3rd, 2011 at 3:06 pm
sanjay: http://www.plosone.org/article/info:doi/10.1371/journal.pone.0010068
November 3rd, 2011 at 5:29 pm
Stapel’s sociopathic behavior has provided a fine excuse for anyone ever done wrong by social psychology to malign the whole field. Got an inferiority complex because you’re in a social science that doesn’t collect experimental data? Pile on. Resent the attention social psychology gets because it’s interesting to people? Pile on. Got a brain fetish? Pile on. Go reductionism!
There are data about outright fraud, and psychology is not even close to the main culprit: http://ori.hhs.gov/misconduct/cases/
The sentiment that subtle biases and systematic incentives are a particular problem for social psychology as compared to other disciplines may be the most laughable of all the schadenfreude-induced stone throwing. Yeah, scientists in other fields aren’t self-interested and are only driven by an earnest pursuit of truth. Cold fusion anyone?
Y’all are sad.
November 3rd, 2011 at 7:05 pm
#13, has anyone done a psychometric study on social psychologists? are you all so defensive and dumb? of course scientists in physics aren’t better scientists than psychologists. the field keeps them “honest.” the soft sciences are hard, and the hard sciences are easy. i never said “scientists in other fields aren’t self-interested and are only driven by an earnest pursuit of truth.”
i don’t agree with the neuroscience-uber-alles opinion expressed above fwiw. reducing all of psychology to neuroscience is about as useful as reducing all of biology to chemistry.
November 4th, 2011 at 12:11 am
God, I’m glad you said that (about neuroscience-uber-alles) because I was ready to do a derisive snort again. I tried to find your 10 questions for Joseph LeDoux (ok, not very hard), but if I reacall right, that is where he said that for fear-conditioning there was already good, stable behavioral paradigms – which was needed in order to understand the neural underpinnings. And all the people I know in neuropsychology spend a lot of time on the behavioral paradigms.
There has been really good discussions going on all over the place on how to deal with this, and specifically what it would be about the system as it is now that makes it vulnerable to this kind of exploitation (or possibly how to make the system more robust so that the inevitable cheaters make minimal damage, and also so we don’t succumb to all the biases we spend so much time uncovering and exploring).
Sanjay linked in once again his commentary on JPSP’s refusal to publish failure to replicate Bem’s ESP studies, and how, in extension, it is a real problem for the field that replications are not rewarded (failed or otherwise – unless they are sexy extensions). And, that is a real problem in a field where there are so many mediators that make the research less robust. (Who did I get that from? I don’t remember, but not original to me, but to the discussions on facebook and google+ that I have participated in).
Just my own experience (and I’m just a lesser researcher) is that when I do more social psychology experiments, the results are that much more fragile than when I do the psychophysical type, and in addition way more difficult to interpret. Instruments don’t travel well between western countries, and sometimes not even outside student populations (and my recent experience with data collected in Thailand).
The failures to replicate, the unexpected variations of results are important to understand social psychological phenomena, yet you rarely see it (I can only recall 2)
But, on the other hand, some interesting and useful things do come out of the field. I teach a class on applied persuasion, based on Cialdini’s book Influence. And just yesterday one of my students told me how much more money his band got in on merchandize from simply applying some of those principles.
(Sorry – I do feel so ranty about this).
November 4th, 2011 at 12:06 pm
Very professional, Razib. I quote: “Here’s the controversial thing I’m going to say: instances of flamboyant fraud are probably far less of an issue in a discipline like social psychology than more subtle biases and systematic incentives.”
Why is that “controversial?” If you don’t mean to imply anything specific about social psychology, then why do you say “a discipline like social psychology?” How about the title of your blog post?
You need an editor nanny.
I’m not defensive, I’m offensive. It is absurd for Stapel’s singularly sociopathic behavior to be taken as a reflection on an entire science, and I’m not going sit quietly when “journalists” make that case.
November 4th, 2011 at 2:57 pm
#16, dude, stop being an asshole. i’m not a “journalist.” and don’t bother follow up commenting, i’ll just delete you.
November 4th, 2011 at 7:32 pm
Dr Jeff Sherman is a frequently published social psychologist whose work is on many of the same topics as that of Diederik Stapel, according to Google.
In how many of your published papers do you cite the work of Stapel? What plans do you have to qualify any of the citations to Stapel in your work?
November 4th, 2011 at 9:56 pm
i’m not a “journalist”
Of course you are a journalist. Who do you think you are?
November 4th, 2011 at 11:02 pm
i’m an internet hobbiest. do you get to determine who is, or isn’t a journalist?
November 5th, 2011 at 9:52 am
i’m an internet hobbiest
I don’t think so. “Hobby: n., an activity or interest pursued for pleasure or relaxation and not as a main occupation.” Since you yourself said that most of your income is related to your blogging (but not directly due to it), it seems that hobby is a poor description. Hobby can become a job, nothing’s wrong with that.
do you get to determine who is, or isn’t a journalist?
Why not? It’s one thing what one thinks of himself and another what others do of him. Surely for someone who outputs for a wide audience, that audience’s perception is more relevant. IIRC, last time you brought up this topic, most commenters thought that what you do *is* journalism.
November 5th, 2011 at 10:16 am
“A journalist collects and distributes information about current events, people, trends, and issues.”
Razib does a lot more than just collect and distribute information.
If he says he’s not a journalist, then he’s not.
Maybe I missed that you were taking a vote last time. So I’m voting this time.
He’s not.
November 6th, 2011 at 8:47 pm
>>Paul Adams Says:
“Social psychology” in the absence of a good understanding how the brain works is like alchemy without physics. Unfortunately the Madoff/Goldman-Sachs analogy is a good one: Stapel committed outright fraud but the main problem is the horde of psychologists that publishes real, but uninterpretable, data. If you are interested in the mind, study the brain.<<
Ignorant, bigoted bullshit. This is tantamount to saying that Darwin or Mendel could not possibly have made any useful contribution to biology because they did not understand DNA. If science had followed this principle we would still be attributing thunderstorms to the wrath of Zeus.
Your chances of learning anything meaningful about what the brain does, if you don't actually know some psychology, are close to zero.