Social Priming: Time for A Definitive Test?

By Neuroskeptic | July 30, 2015 3:02 pm

The scientific controversy over social priming – the (claimed) ability of incidental exposure to stimuli to ‘prime’ or evoke complex behaviors – has been rumbling for the past few years. Will it ever end?

In this post I’ll propose one way in which I think it could be resolved.

‘Social priming’ is a fascinating concept. A paradigmatic example of a social priming effect is the claim that solving word puzzles involving words such as ‘old’ and ‘elderly’ makes people walk slower, presumably because these words ‘prime’ the concept of old age, which stereotypically means slow movement.

fixing_science

Social priming has been around for over 20 years. The current controversy dates back a few years; the rise of social priming doubts began, I think, in 2012, a year in which a group of researchers could not replicate the old people-slow walking effect, leading to some acrimonious blog posts by John Bargh, who had originally reported that effect in 1996.

2012 was also the year in which Diederik Stapel was found to have committed academic fraud on a massive scale, many of the studies that he fabricated involved social priming. Social priming is not the only area of psychology involved in the ‘replication crisis‘, but it’s fair to say that this is where it has been most acute.

And yet, as far as I can see, this debate has not achieved much. The battle-lines are well defined and I don’t see many people who have changed their minds in either direction. Each new study is embraced by people who already agree with its conclusions, and written off by those who disagree. At times the debate has led to heated personal attacks.

Is there a better way? I think there is.

Rather than a tug-of-war in which believers and skeptics are pulling from different sides, someone should organize a collaborative team, involving leading representatives of both parties: an adversarial collaboration. This alliance should then aim to carry out the largest ever study of social priming effects, with the goal of a definitive answer, one way or the other.

The proponents would be asked to select a handful of social priming effects that they consider the most likely to replicate. They would also be asked to name (within reason) the methodological parameters that they consider important to success. The skeptics would participate in these discussions, and then everyone would decide upon the protocol. All of the methods and data analysis plans would then be publicly preregistered, to prevent p-hacking by either side.

The actual data collection would then be done jointly by the believers and skeptics (perhaps spread over multiple labs). All data would be sent to a central repository. Any unforeseen problems arising would be resolved by a joint decision of both sides. Finally, once all of the data was collected, the results would be made public and analyzed in accordance with the agreed-upon plan.

Before embarking on data collection for the study, all of the researchers would be asked to state that they have confidence that the planned study is scientifically valid, and that it will be a fair and genuine test of the social priming hypothesis. In this way, the involved researchers would be committing themselves to accept, or at least not lightly reject, the results that they would be about to help collect.

Perhaps this statement of intent would be the most important part of the project, because it would make it more difficult for the ‘losing’ side to write off the study as poorly designed. This is also why it’s important that both believers and skeptics are represented in the planning and execution of the study. The fact that the protocol and data were all posted publicly would also help increase the credibility of the results.

As to how one could organize such a thing, that’s another question. But it would be a sensible use of resources. Lots of money has been spent, and continues to be spent, funding social priming research; it would be a waste if the whole line of research just petered out eventually without any truly convincing results ever emerging.

Whether or not social priming exists is a very interesting question. No amount of small experiments will provide a definitive answer: there have been hundreds already, and look where we are. A large, carefully controlled, and collaborative study is needed.

ADVERTISEMENT
  • https://plus.google.com/u/0/101046916407340625977/posts Rolf Degen

    In my view, the main problem with social priming is that it doesn’t make sense from a theoretical standpoint. It presumes that we are built by evolution in a way that makes us susceptible to random environmental influences beyond our control and not useful in an adaptive way. But if it existed – if analogous behaviors were triggered unconsciously by respective stimuli in the outside world – there would be such an excess of subtle triggers in real life that they would cancel each other out. https://plus.google.com/101046916407340625977/posts/4p4xrKP7eC8

    • D Samuel Schwarzkopf

      That’s kind of the discussion I had in response to this post on Twitter as well. I agree in that the lack of a theoretical basis for how such (often rather strong) effects could really manifest without being horribly diluted by all sorts of confounding factors (even under lab conditions) is what is bothering me the most about social priming.

      But to be fair, I think some things are perhaps more reasonable than others. I don’t think a tiny flag in the corner of the screen is going to influence voting behaviour several months in the future. But it seems more reasonable that being exposed to certain words and concepts or other contextual stimuli could affect autonomous behaviours (like walking) in some way.

  • OWilson

    Phew! That’s a lot of work, or research money.
    Why not jut interview a Madison Ave advertising type?.
    He can tell you about how shopping center music is chosen, the average reaction to the words, “new” and “improved” and “classic”.
    Not to mention how beautiful models can sell cars.
    I thought this stuff was old hat.:)

  • Nick

    Pedantry Corner: Stapel was actually unmasked (and fired) in 2011. The committee that investigated his fraud reported in late 2012, and many of his retractions date from that year.

    On the adversarial collaboration, I’m skeptical. In order to organise one, it would be necessary for there to be good-faith agreement between the two parties that the main problem can be solved with more data. Given that the reaction of the people in the original lab is often “These people have impugned our integrity!”, while on the other hand in a few cases the (non-)replicating lab are often a little too gleeful (perhaps not in the article, but check their social media) about just how big an apple-cart they’ve upset, I’m not sure how likely it is that the necessary dialogue can take place. (Plus, who would apply for the funding, how likely would it be that they’d get it, and if it didn’t get accepted by the original journal, how big a hit would everybody’s personal impact factor average take?)

    Without this trust, you could only get such an adversarial collaboration going with the supervision of a neutral third party, which means that at least one party is going to basically have to be dragged kicking and screaming into the lab. I’m going to guess that they are not then going to accept a possible adverse result meekly. You would see all kinds of arguments about things you didn’t believe people could disagree on.

    Let’s face it, many researchers (not just in priming) have a substantial intellectual (and in some cases, career or financial) commitment to their stuff being true. We know (from psychology, ha ha) that under those circumstances, people will do a great deal of twisting and turning to avoid abandoning their cherished beliefs. Of course, in the process, something that they sold to the Daily Mail as “true for everyone, with an effect size greater than smoking” might have to become “true for Yale students but not Harvard students” for a week or two, until the fuss calms down, but if there’s no actual retraction, well, you can still cite it, right? In this regard, a re-reading of Meehl’s 1967 classic “Theory-testing in psychology and physics – A methodological paradox”, starting at the bottom of p. 113, is a worthwhile and entertaining use of three minutes.

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      I agree that “In order to organise one, it would be necessary for there to be good-faith agreement between the two parties that the main problem can be solved with more data.” However, I don’t think that this would be impossible – although it might be hard.

      Both sides of this debate are conducting studies and meta-analyses of studies to support their claims. Both sides are generating data.

      All that needs to happen is for them to agree to generate data together.

      Now in the case of (say) psychoanalysis, where most of the proponents do not believe that their theory can be or should be tested experimentally, then I don’t see how a definitive test could be arranged.

      But social priming is not like that.

  • Alexander Etz

    I imagine even with the pre-registration and statements of intent, if effects are not found then the people whose careers have been built on this will find a way to deny the reality once again. Motivated reasoning is not blocked by simple pre-registration or the like.

    Just look at the adversarial collaboration that Dora Matzke and colleagues did. Did that change anyone’s mind?

    • D Samuel Schwarzkopf

      My view on adversarial collaborations is that it doesn’t really matter what the researchers themselves think. Most adv collabs I’ve seen ended with the two parties still disagreeing. The thing is though that the evidence is usually relatively clear and so the wider readership can make up their own minds. I think this is more important than whether individual researchers change their minds.

      • Nick

        I’m reminded of what apparently happens regularly in carefully designed tests of dowsers. One of the things the researchers do is to allow the dowsers to explore the field where the tests are to take place, and satisfy themselves that there are no external influences that might block their magical ability (say, a visible power line). After much negotiation, typically including changing the colour of the researcher’s sweater from red to blue, the dowsers declare themselves satisfied, and the tests commence. Of course, the dowsers fail, and of course, they always come up with an explanation that they forgot to mention beforehand, such as the colour of the researcher’s socks (“Terribly sorry, I only just noticed now that they’re white, it disrupts the energy field”).

        Of course, this is just an amusing anecdote of a slightly analogous situation in pseudoscience. Proper scientists with PhDs would never behave like this.

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      I don’t know if the Matzke collaboration changed minds (I blogged about it here BTW).

      However, that study was not especially large; their sample size (quoting from the paper) “closely approximates the median sample size (i.e., 25 participants per condition) used in between-subjects free recall studies in the relevant eye movement literature.”

      It might have been more definitive had it been, as I’m suggesting the social priming one should be, the largest ever study in the field.

  • D Samuel Schwarzkopf

    In some ways this is already what some of the Many Labs experiments are about, isn’t it? Except in those cases they don’t seem to be adversarial collaborations (although they probably involve people with a range of beliefs in the truth of these effects). I think it’s a good idea although not sure how realistic it is that it will happen…

    • T.

      I heard they’re going to perform a Registered Replication Report on the “professor priming” study.

      If i understood things correctly, this means a lot of what is proposed in the above article will be executed: design of a protocol with help of the original author, to date it will be (one of) the largest experiments on “behavioural priming”, it will be pre-registered, and it will be performed by multiple labs.

      • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

        That’s good news. But social priming advocates could still say that *other* forms of social priming are valid even if professor priming is not. This is why I think we should get the whole community to select the effects that they consider most replicable (the “champions” who they think should represent them in “battle”) and then we could test those.

        • T.

          I think this has been done:

          “Pashler issued a challenge masquerading
          as a gentle query: “Would you be able to suggest one or two goal
          priming effects that you think are especially strong and robust, even if
          they are not particularly well-known?” In other words, put up or shut
          up. Point me to the stuff you’re certain of and I’ll try to replicate
          it. This was intended to counter the charge that he and others were
          cherry-picking the weakest work and then doing a victory dance after
          demolishing it. He didn’t get the straightforward answer he wanted.
          “Some suggestions emerged but none were pointing to a concrete example,”
          he says.”

          source: http://ktwop.com/2013/05/11/how-much-of-social-priming-psychology-is-just-made-up/

      • D Samuel Schwarzkopf

        Ah yes professor priming. Thanks for primi… reminding me ;). That’s another one I find mechanistically implausible.

  • unity_ministry

    Without wishing to pour cold water on your idea, I rather suspect that even a large, carefully controlled, collaborative study is unlike to settle because I’m not sure it would ask the right questions.

    My feeling here – and it is really a feeling – is that the problem with social priming isn’t a matter of whether it does or doesn’t exist – because I think it probably does – but that it’s likely to be:

    a) a relatively transient phenomenon, which raises serious problems in terms of both replication and ecological validity – you may be able to demonstrate an effect in a strictly controlled experimental environment but in real world settings, where there are a multiplicity of potential priming influences, it will be impossible to predict which, if any, will generate a measurable effect in any given situation, and

    b) a phenomenon that relies on cultural associations which almost certainly aren’t going to be valid for all people at all times. These experiments make assumptions about stimulus-response associations which may not only be invalid between different cultures but also within cultures, so if you do happen to strongly associate “old” with thoughts of your arthritic grandmother than maybe you do unwittingly end up walking more slowly but if you went to see the Rolling Stones a couple of days ago and you are, in some sense, already primed to associate “old” with Mick Jagger bounding around the stage then you’re unlikely unconsciously respond in the same manner.

    That being the case, I’m really not sure a definitive test is possible certainly not to the extent that would establish that priming is a socially significant phenomenon.

  • Umair Itrat

    My hunches tend to be confirmed by “studies”. I think the effect of social priming on individuals varies significantly and even what primes them. The effect may be subtle and small and thus difficult to replicate or analyse.

    I am not sure if the scientific community can agree to conduct the research on a large enough sample size and with large enough repetitions / variations of priming to conclude decisively one way or the other.

  • http://www.mazepath.com/uncleal/qz4.htm Uncle Al

    Everything in psychology is both true and untrue. Like 45 years of quantum gravitation and SUSY, psychology must be true even if it empirically is not. “A large, carefully controlled, and collaborative study is needed.” It’s been done: chief advisor to King Théoden of Rohan, Gríma Wormtongue.

    WWI and nascent WWII were in crisis. City boys did poorly before withering fire (“shell shock”) and mass defecation in muddy fields. Psychiatrists offered years of “analysis.” The solution? The psychologist, who would exercise authority as ordered, “send them back into the meat grinder,” or be sent in their stead. Like Southern Baptist ministers keeping slaves docile “toward a greater reward,” the salesmen began believing their own sales pitch.

    • OWilson

      For millennials “truth” is whatever doesn’t offend you.

      • http://www.mazepath.com/uncleal/qz4.htm Uncle Al

        One supposes hammering in a nail is still a grey area, unlike earned wages versus deserved support.

        • OWilson

          Is actually neuro-motor skills which cause the regular pattern of hammering? Or does it have something to do with arrythmiatic predilection?

          I’m not aware of any recent studies that show a dissonant pattern of hammering drives the nail deeper vis a vis an regular interval between strikes.

          • http://www.mazepath.com/uncleal/qz4.htm Uncle Al

            Said Social Prime Rib: 18 August 1978, Theodore Streleski plus a small sledge hammer, Professor Karel de Leeuw, Stanford University. I claim systematic and statistical faults for not evaluating a variety of hammers (including Youtube v=i-ahuZEvWH8) and faculty members (e.g.,Hotel Management, Law, Kinesthesiology).

  • empireinrecline

    scientific research has no room for “believers” or “skeptics” it’s either good experimental design or it’s not. You use LOGIC to set up an experiment, collect the data, and analyze it. Period. Social sciences have always been suspect because of the ‘softness’ of the data. I’m a biologist, if I can’t point to my result that I have achieved repeatedly and is replicated in another set of hands, it isn’t real. So this clusterfuck of subjective data, ego, and money grubbing muddies the scientific waters, but it isn’t surprising.

  • Jeff Sherman

    The term “social priming” is meaningless. Yes, the subtlety of the prime
    matters, but the DV is the more important source of unreliable results.
    The studies that are difficult to replicate often have a single behavior
    as the DV (walking speed). In contrast, “social priming” studies that
    collect data from hundreds of trials (evaluative priming, semantic
    priming, the Weapons Task, the Shooter Task, the IAT–which is more or
    less a priming task) produce robust findings that are easily replicated.
    And, that is true even when the primes in those tasks are presented
    very subtly.

    So, what this all boils down to is that effects measured with a single
    response are less reliable than effects measured with many responses. It
    has nothing to do with “social” or even “priming,” per se. If Bargh’s
    Ss walked down that hallway 250 times, the results would be more robust.

  • lump1

    I’m not optimistic that people could unite to do something so reasonable, but it would be great. Somebody needs to set this precedent as a way of solving empirically testable questions in science.

  • Anonymouse

    Without a theory of priming in general, which would allow not only to predict the direction, but also the magnitude of the priming effect of a given stimulus, replicability faces a very general problem – why should things replicate anyway?

    Language is a great example, because it’s probably the most important source of stimuli and because those stimuli are well quantifiable by the use of corpora. Take for instance the experiment with words that supposedly prime the concept of “old people”:

    It’s not an uneducated guess that what perceiving a word does is change one’s expectations about upcoming events (sort of the point of having a brain). So looking at which words are most likely to follow each of the primes used in that experiment could be fairly informative about what kinds of concepts those stimuli will actually prime*.

    Looking at a corpus that comes with meta data about time like COCA (Corpus of Contemporary American English) allows to actually see how the distribution of words likely to follow each stimulus has changed over time. Now if you, say, were look at the 10 most likely words to follow each prime (that will account for the bulk of the instances due to language’s zipfian distribution) and you found that the distribution has dramatically changed, such that the experiments’ primes aren’t followed by words related to the concept of old people as often these days – why would you expect this the behavioural observation to replicate? I don’t have the reference for it at hand now, but that is exactly what has happened in this experiment.

    Yet the view that conceives of language as something static forbids this kind of thinking (#whodoesntlovechomsky). But it should, by now, be clear to everyone that a person’s language differs dramatically from one person to the other, especially with spatial and temporal distances, and that the idea of static meaning (and thereby constant priming over time) is untenable. So linguistic stimuli need to be controlled, rather than chosen according to a researcher’s intuition. As long as that doesn’t happen, who knows what’s going to replicate and what isn’t. And who cares, really.

    The point is the same for non-linguistic stimuli, except that they might be much harder to quantify. But again, expecting that a red cape will prime the concept of a bull and lead to people that participate breathing more heavily on the way back to the elevator, independent of time and place of the experiment, is silly at best.

    Anyway. I think what you’re proposing is a good idea. But then again people don’t really change their minds even in the face of evidence that contradicts their opinion, do they? (;

    * In that view “priming” is simply how processing the prime will favour subjects’ expectation of the target stimulus or the target bahaviour as compared to the baseline (which comes with the same lack of justification).

  • Pingback: I’ve Got Your Missing Links Right Here (15 August 2015) – Phenomena: Not Exactly Rocket Science()

  • Pingback: I’ve Got Your Missing Links Right Here (15 August 2015) | Gaia Gazette()

  • Pingback: Do Bilingual People Have a Cognitive Advantage? - Neuroskeptic()

  • Pingback: "Troubling Oddities" In A Social Psychology Data Set - Neuroskeptic()

  • Pingback: "Troubling Oddities" In A Social Psychology Data Set – Discover Magazine (blog)()

  • Pingback: The sorry state of modern psychology — Science of the Spirit — Sott.net – Your Libertarian NewsCast()

  • Pingback: Undead theories: The sorry state of modern psychology | Fake News Registry()

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Neuroskeptic

No brain. No gain.

About Neuroskeptic

Neuroskeptic is a British neuroscientist who takes a skeptical look at his own field, and beyond. His blog offers a look at the latest developments in neuroscience, psychiatry and psychology through a critical lens.

ADVERTISEMENT

See More

@Neuro_Skeptic on Twitter

ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Collapse bottom bar
+