I have a new feature out in Nature looking at two big problems within the field of psychology. First, the field is almost entirely dominated by positive results, while negative ones languish unpublished in personal file drawers. Second, there are few incentives to replicate old results and negative replication attempts face a lengthy gauntlet of obstacles. In the story, I look at why these problems exist and why some psychologists are starting to take them very seriously.
The piece has its origins in an incident that regular readers will already know. In January, Stephane Doyen and colleagues had unsuccessfully tried to repeat a classic experiment where people walk more slowly down a corridor after being unconsciously primed with age-related words. I wrote about their research. Two months later, the man behind the original study – John Bargh of Yale University – wrote a scathing attack on Doyen’s team, me, and the journal that published the study. I responded.
The ensuing discussion opened my eyes to an undercurrent of unrest. Many psychologists came out of the woodwork to mention experiments that were hard to replicate, common practices that they deemed to be dodgy, and a growing willingness to turn a critical eye upon their own field. For every comment that appeared on the blog and Twitter, I’ve got another that was sent confidentially to me via email. This was clearly something worth writing about.
From the outset, I was very clear that this was not going to be a hit piece, against any individual researcher, or psychology as a field, and I’d like to think that the result is a fair assessment. I look at issues such as: conceptual replication (where different but thematically similar experiments are said to replicate each other); the difficulties of repeating studies of abstract concepts; the role of almost magical ‘knacks’; and how tiny, innocuously-made choices that pull statistically significant results from the ether. I also look at the extent to which these problems are unique to psychology, compared to other sciences.
And perhaps most importantly, I look at the triggers that have provoked many psychologists into recognising and discussing these problems, from controversial papers about psychic abilities to high-profile frauds like Diederik Stapel. It is not my contention that such misconduct is common but I think that the field’s cultural norms have created an environment where Stapels could go unnoticed.
But enough background. Do read the piece, and let me know what you think. I might post a bit more about it, depending on reactions.
I’m going to collect some other good resources on this issue:
- An entire special issue on the topic from The Psychologist
- Psychology’s bold initiative – great piece from Siri Carpenter about Brian Nosek and his Reproducibility Project
- Navel gazing on steroids in social and personality science - Brent Roberts summarises some of the recent events, including some I didn’t touch on in the piece
- Bias in psychology – Razib Khan riffs off my piece.