Does Psychology Need SWaG? The Ethics of Naturalistic Experiments

By Neuroskeptic | January 21, 2018 10:59 am

Diederik Stapel. Brian Wansink. Nicolas Guéguen. Anyone who’s been following recent debates over research integrity in psychology will recognize these as three prolific and successful academic psychologists who have suffered a total (Stapel) or ongoing (Wansink, Guéguen) fall from grace in the past few years.

If you’re not familiar with these cases, you can start by reading over Nick Brown’s blog. Brown has been at the centre of the investigations into irregularities in Wansink and Guéguen, and he also translated Stapel’s book into English.

In this post, I will not revisit the issues with these three psychologists’ work. Rather, I want to point out that, taken at face value, their work was brilliant. They published research that, if real, would have been a model of what experimental psychology should be.


Stapel, Wansink and Guéguen – let’s call them SWaG – worked on different topics, but what all of them had in common was that they published naturalistic experiments.

Conventional psychologists carry out experiments, but not naturalistic ones. They invite people (students mostly) to come into a testing room and perform some tasks or assessments in return for money or course credit. It’s an artificial environment. The participants know they are being studied. It’s not natural, not real-life.

SWaG’s experiments, by contrast, were realistic (if not necessarily real). Guéguen, for instance, reported sending a female investigator into real bars to see how long it took for real men to hit on her, depending on the heels she was wearing, which varied. A silly topic perhaps, and the study may have been fabricated, but setting those issues aside… it’s great. It’s an application of the experimental method to actual human behaviour in ‘the wild’.

I don’t think that a conventional lab psychology experiment would be able to tell us whether high heels help women to get hit on. Lab studies are well-suited for the study of perceptual or cognitive processes, but for complex, social behaviours, I’m not convinced that the lab is the right place to experiment.

My point is, maybe psychologists should look at SWaG and “do as they say, not as they do”. Don’t do what they did, but try to actually do the kind of things that SWaG claimed to have done.

Of course, it’s easy enough to say that experiments should be naturalistic. Naturalistic validity is difficult to achieve in practice. Perhaps the most fundamental problem is ethics: by definition, a truly naturalistic study would involve experimenting on people without their consent, because if they knew they were consenting to take part in a research project, it would no longer be a real-life situation. What ethics review board would approve such a project?

It is unclear whether Guéguen got official ethical approval for his studies – although if these studies never actually happened, I suppose he didn’t need it. But a legitimate naturalistic experimenter would need it, in most parts of the world (although maybe not in France, where Guéguen works.)

Perhaps there is a lesson here for our ethical standards. The fact that SWaG’s work was highly cited shows that naturalistic work is valued by the research community, and – as far as I know – there was no public outcry over the ethics of these kinds of experiments, at least not until the scandals started. Yet it seems that it is very hard to do these kinds of studies without resorting to SWaG-like shortcuts. Perhaps we need a more open-minded approach to the ethics of naturalistic experiments?

CATEGORIZED UNDER: ethics, methods, science, select, Top Posts
  • Nick

    Something I’ve wondered about the Cornell Food and Brand Lab: reports (including from neutral observers, such as Tom Bartlett in his 2017 piece in the Chronicle) talk about kitchens and eating areas with two-way mirrors and hidden cameras. Maybe I’m missing something, but apart from the bottomless soup bowl study, which I think from memory is at least 10 years old, we don’t hear very much about all the cool discoveries that are being made with all these expensive facilities.

  • Pingback: Does Psychology Need SWaG? The Ethics of Naturalistic Experiments – Medicine by Alexandros G. Sfakianakis,Anapafseos 5 Agios Nikolaos 72100 Crete Greece,00306932607174,00302841026182,

  • Pingback: Does Psychology Need SWaG? The Ethics of Naturalistic Experiments – Iatrika Nea by Alexandros G. Sfakianakis,Anapafseos 5 Agios Nikolaos 72100 Crete Greece,00306932607174,00302841026182,

  • Uncle Al

    depending on the heels she was wearing” Unless it was a Lesbian bar, one doubts assaying gazes traveled much south of her sternum. Guys like discovering Easter eggs.

  • Daniel Hackett

    It shows there is no integrity in the field whatsoever. The work is highly cited because they told a good story. You don’t seem to realize that it’s all about getting a headline for clickbait. These stupid studies are not about psychology, they don’t advance our knowledge, and don’t help people with problems. They are just any old rubbish which generates ad revenue. Most research is about dollars now, not sense.

  • Pingback: Naturalistic experiments – What is behavioral? A blog of recent updates to behavioral economics()

  • Kingston Chi

    When you go for ‘real world’ [naturalistic] experiments, don’t you run into the infamous ‘Chinese herbal medicine’ effect ? where even if you know the herbal soup you drank cured you, you do not know what ingredient actually did it, or how much or little would have done a better job ..

    • jrkrideau

      Yes and no. You might have to go for an extended series of studies with systematic manipulation of individual variables and look for converging lines of evidence but you could do it in many cases. You might need to move a quasi-experimental design methodology.

      In other situations, depending on the research question, getting ethics approval could be very dicey.

      Here is a press report on one slightly wacky but very interesting field study.

      Sorry I don’t have the reference handy. It would need to be replicated and I don’t know if it has but it was an interesting field experiment and done, probably, at little or no cost.

      However, as some of Nick’s discussion points out, field research can get expensive, both in time and money very quickly. That may be one major reason that many people stick in the lab.

      I once worked as a student research assistant where I was in the field for four month, about 1,500km from the office. This starts to get costly.

      If you need to enlist the co-operation of local groups, just the time to contact them, explain the project and get agreement may run into months.

  • PsyoSkeptic

    Regarding ethics, studies like the high heels one could be conducted under the new guidelines with no review whatsoever. People aren’t being experimented on. People were doing whatever they would naturally do in a location they volunteered to go to in public and it was observed. She’d have to be dressed in a way outside what would normally be encountered in the situation for any ethics to come into play.

    You could do the less controlled study of just observing both number of hits and heel height without ever controlling for the wearer. Maybe that would come out in the wash. And wouldn’t it have the same problems as the controlled study since the fundamental confound, natural proclivities of those who wear different heel heights, are unknown by the hitters for the particular woman in question but would have impacted their learned associations.

  • PsyoSkeptic

    Weren’t Stapel’s studies primarily in lab? And wasn’t the main issue with Wansink that everything was just a fishing expedition due to the naturalistic environment and large number of variables that could be explored (but talked about hyothetico-deductively)?

    None of these guys are models for science independent of the fraud. I think you’ve been drawn in by the interesting stories.

    • Neuroskeptic

      Some of Stapel’s studies were in lab but others were in the field, e.g. (supposedly) he sent a black experimenter and a white experimenter to a train station and saw how close people sat to them.

    • Neuroskeptic

      Wansink’s studies were fishing expeditions but I think this was more a problem with the analysis than with the actual study designs. AFAIK they did have a priori hypotheses, but often they weren’t confirmed, leading to fishing (see e.g. The Grad Student Who Never Said No)



No brain. No gain.

About Neuroskeptic

Neuroskeptic is a British neuroscientist who takes a skeptical look at his own field, and beyond. His blog offers a look at the latest developments in neuroscience, psychiatry and psychology through a critical lens.


See More

@Neuro_Skeptic on Twitter


Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Collapse bottom bar