How Blind is Double-Blind?

By Neuroskeptic | March 24, 2010 10:20 pm

There’s a rather timely article in the current American Journal of Psychiatry: Assuring That Double-Blind Is Blind.

Generally, when the list of the authors’ conflicts of interest (550 words) is nearly as long as the text of the paper (740 words), it’s not a good sign, but this one isn’t bad. Perlis et al remind us that if you do a double-blind placebo controlled trial:

The blind may be compromised in a variety of ways, however, beginning with differences in medication taste or smell. Of particular concern may be the emergence of adverse effects, particularly when those adverse effects are known to be associated with a specific medication … Indeed, when the degree of unblinding is assessed in antidepressant trials, multiple reports suggest that it is extensive: at least three-quarters of patients are typically able to correctly guess at their treatment assignment.

The point of a placebo-controlled trial is that neither the patients nor their doctors know whether they’re getting the placebo or the real drug. Hence the strength of the placebo effect should be the same in each group, allowing the “real” drug effect to be measured.

But if the drug causes side effects, as pretty much all do, then people could work out which group they’re in by noticing whether they’re feeling side effects or not. This might enhance the placebo effect in the drug group, and make the drug seem to work better than it really does. Or it might not. But the possibility that it might is worrying.

This is called the active placebo effect. It’s why I’m skeptical of claims that scopolamine and ketamine have rapid-acting but short lived antidepressant effects. I may be wrong, but while both of these drugs have been shown to work better than placebo, both have very pronounced subjective effects, so there’s no chance the blind will have been intact.

Whether the active placebo effect also underlies the efficacy of established antidepressants like Prozac is very controversial. There have been 9 trials comparing antidepressants to active placebos, i.e. drugs that have similar side effects to antidepressants and that should therefore help to preserve the blind. (The active placebos were all atropine which is basically the same as scopolamine.)

The trials were reviewed by antidepressant critic Joanna Moncrieff et al who found that the overall effect size of antidepressants vs. active placebos was d = 0.39. That’s not very high, although it’s not too bad, and ironically it’s actually higher than the effect that Moncrieff’s friend and fellow Prozac-baiter Irving Kirsch found in his famous 2008 antidepressant vs. sugar pill placebo meta-analysis, d=0.32. So if you take that seriously, the active placebo effect plays no part in antidepressant efficacy. However the active placebo trials are mostly small and old, so to be honest, we don’t really know.


A point that’s often overlooked is that a drug could have an active placebo effect via having a “real” psychoactive treatment effect. Diazepam (Valium), for example, has basically no peripheral side effects at all: unlike scopolamine it doesn’t cause dry mouth, nausea, etc. But it is a tranquillizer; it causes calmness and, at higher doses, sleep. They’re pretty noticeable. So if you were to give a depressed person Valium and tell them that it’s not only a tranquillizer, it’s an antidepressant, then the active placebo problem would arise.

In fact, any active drug will also produce active placebo effects – almost by definition, if you think about it. These may be hard to disentangle from the “real” effects. Say you’re anxious about giving a speech so you take some diazepam hoping to feel calmer. A short while later you feel the lovely warm tranquillizing feeling setting in. Phew, you’re calm now, anxiety’s gone, the speech will be no worry. That thought might well be tranquillizing in itself. In other words the anti-anxiety effects of diazepam are partially driven by active placebo responses due to… the anti-anxiety effects of diazepam.


This leads onto another point. Suppose a drug has a genuine effect which improves some of the symptoms of a disease. Does that drug “treat” that disease? In a weak sense, yes, and it might be a helpful drug, but it’s not a specific treatment. Morphine’s very helpful in cancer, because it treats pain, but it doesn’t cure cancer. Likewise insomnia is a symptom of depression, but we feel that in order to qualify as an antidepressant a drug has to treat the core symptoms: mood, anxiety, etc. rather than just being a sleeping pill.

But suppose someone suffered from low mood and you gave them a treatment which stopped them feeling any moods or emotions. That solves their low mood problem: no mood, no problem. But is that a specific treatment for depression? It’s a bit of a grey area, but many would say no.

Many people say that this is exactly what SSRI antidepressants do: they blunt your emotions. That doesn’t mean they’re not helpful in depression: a lot of people find them very useful. I did. But then are they really “antidepressants”, or just anti-mood? SSRIs are the drugs of choice not just for depression but also most anxiety disorders, and obsessive-compulsive disorder, etc. In fact they work better in OCD than they do in depression, relative to placebo. So are SSRIs actually antiobsessives that happen to be helpful in some cases of depression? Good question.

Here’s Chris Rock on the issue of non-specific effects…


Perhaps an ideal clinical trial of a drug for a psychiatric condition should have 4 groups: the drug you’re studying, another psychotropic with non-specific effects (e.g. Valium, or caffeine if you want a stimulant), an active placebo with purely peripheral side effects, and sugar pills. But even then, if the active drug performed better than the other 3 groups, a die-hard skeptic could say that maybe it’s just more effectively causing non-specific sedation, or blunting, or whatever, than the Valium. Ultimately, a randomized controlled trial can never prove that a psychotropic drug has a specific as opposed to a non-specific effect.

So where do we stand? Does that mean we don’t know what drugs do? No – unless we’re some cloistered soul who only reads papers as opposed to talking to people, reading subjective reports, or taking drugs themselves. I know what alcohol does, not because I’ve read papers about it, but because I’ve drunk it. I’ve also been depressed and taken antidepressants, and for what it’s worth, in my experience, some of the drugs currently marketed as antidepressants do have a specific anti-depression effect, although others don’t. Overall, though, my view is that we know surprisingly little about what antidepressants actually do.

ResearchBlogging.orgPerlis RH, Ostacher M, Fava M, Nierenberg AA, Sachs GS, & Rosenbaum JF (2010). Assuring that double-blind is blind. The American journal of psychiatry, 167 (3), 250-2 PMID: 20194487

Moncrieff J, Wessely S, & Hardy R (2004). Active placebos versus antidepressants for depression. Cochrane database of systematic reviews (Online) (1) PMID: 14974002

  • -p.

    “So if you take that seriously, the active placebo effect plays no part in antidepressant efficacy.”

    Did I misread you? No part?

    Moncrieff's Cochrane Review looked at 9 trials of antidepressants vs active placebos, only 2 of which showed statistically significant effects (Kirsch would argue that even statistically significant effects aren't necessarily clinically significant, and he has a point). Much of the overall correlation found was attributable to a single strongly positive study; removing that study dropped the overall pooled effect to 0.17 (CI reaching 0). The conclusion was: “The more conservative estimates from the present analysis found that differences between antidepressants and active placebos were small. This suggests that unblinding effects may inflate the efficacy of antidepressants in trials using inert placebos.”

    Moncrieff may be a “critic” and Kirsch may be a “Prozac baiter”, but that doesn't make the evidence that antidepressants have an clear effect beyond that of active placebos any stronger.

    I'm entirely sympathetic to the idea that antidepressants might have a specific effect against depression. But that doesn't mean the evidence is any stronger than it is (it's not that there's no evidence, but that the evidence is hardly overwhelming or bulletproof). As skeptics of all tribes like to point out, the onus should be on those claiming an effect to show that it occurs, and how.

  • Anonymous

    Right on! In your face, NP!!!

  • Anonymous

    Given that the real life use of any product will not be blind (both prescribers and users will know what they are using) what's the point with this obsession with “double blind trials”?
    Some murky “philosophy” hiding in the background…

  • Neuroskeptic

    -p.: Right, but my point is, if you take Moncrieff et al using active placebos seriously you get d = 0.39, and if you take Kirsch et al using inert ones seriously you get d = 0.32. Meaning that antidepressants do better against active placebos than inert ones.

    I would class excluding the strongly positive study as not taking Moncrieff et al's data and the d = 0.39 entirely seriously. The fact that Moncrieff et al did that themselves doesn't alter that judgement. Once you go down the road of excluding studies like that you can make a meta-analysis produce whatever results you want…

    Personally I agree that active placebo effects do play a part in antidepressant effects but we can't conclude that from those two studies. The Moncrieff et al studies used first-generation antidepressants, and were mostly done in the 70s and 80s, so I'm not sure they're studying the same “antidepressants” or the same “depression” at all.

  • Jaz

    Without attending a drug alcohol treatment program under the care of medical professionals, you cannot hope to achieve complete recovery.

  • Dirk Hanson

    I don't think the placebo effect probably shows itself that much unless you are testing SSRIs on a population that doesn't need them, i.e. the mildly or situationally depressed. Clinical unipolar depressives, in my opinion and experience, aren't likely to be sustained for very long by a placebo effect, although I agree completely that a true double-blind study is very difficult to pull off.

  • cheese omlette

    “at least three-quarters of patients are typically able to correctly guess at their treatment assignment.”

    You have written before about the nocebo effect that sometimes occurs with side-effects, it’s possible that this occurs in the other one quarter in the control group who incorrectly guess their treatment assignment. What about comparing the people in the control group who think that they have been assigned the active drug with the people in the drug group who correctly guess that they have been assigned the active drug? Surely the placebo effect would be active in both, and therefore it would be possible to compare the two and find out the real effect of the drug?

  • Anonymous

    Hanson you are so wrong. See CLPSYCH blog for more education about the enduring effects of placebo. Educate yourself!

  • bleikind

    Depression is an affliction with potentially fatal outcomes, through suicide.
    Researchers and physicians believe that there are effective, if not ideal, treatments.
    Why is it ethical to conduct placebo-controlled, double-blind studies in which one group of patients receives no treatment?
    In cancer trials, patients in one arm receive the best available therapy, and patients in the other arm receive the best available therapy plus the new treatment.
    Why aren't similar designs required in depression studies?
    This would also help the problem of patients or researchers identifying those in the new treatment arm.
    Also, there is no objective measure of depression. In a cancer trial the researchers measure the progress of the treatment directly without asking the patient.

  • Anonymous

    Hey bleikind:
    So you want more rigorous antidepressant trials and then you say there is no objective measure of depression? So why bother?????

  • bleikind

    I didn't call for more “rigorous” trials of anti-depressant treatments. I called for more ethical ones. Trials for anti-depressant treatments are unethical if they include arms in which patients receive no treatment or only treatments purposely selected because they are ineffective against depression.

    The fact that researchers who study anti-cancer therapies can use objective measures of the disease to assess their treatments, while anti-depressant researchers can only ask questions of their subjects shows why studies of depression are difficult. Most of the troubles Neuroskeptic carefully analyzed in his post result from the lack of an objective measure of depression. All present measures involve asking the sufferer questions about symptoms instead of the researcher making measurements, noting signs, lab tests, MRI tests, and so on.

  • Neuroskeptic

    Measuring depression is extremely difficult. In a sense depression is inherently subjective so there will never be an objective test, but I've often wondered whether meaningful activity level isn't a pretty good quasi-objective measure of mood. If you're going around doing everyday things, enjoying yourself, making decisions, working productively, making plans, having ideas, writing e-mails etc. – you are not severely depressed.

    While depressed, unless it's really extreme, you can function well in specific situations, but the rhythm of daily life as a whole falls apart; you can do things if you literally have to do them, but you lose all initiative and desire to do anything beyond the minimum. Whereas the first sign of (hypo)mania is generally increased activity, making plans, spending money, socializing, working, etc – doing more than you'd usually do.

    So maybe a good way of tracking mood wouldn't just be to ask people, what did you do today? Although whether this would be useful in practice I'm not sure. But I feel it would better than simply asking people Are you depressed? Because the first thing you lose in depression (in my experience anyway) is the ability to recognise that.

  • Dirk Hanson

    From CLPSYCH: “Anyone still want to seriously argue that the vast majority of the antidepressant effect is not the placebo effect?”

    Sure. The positive SSRI effect, though sometimes more modest than everyone would like,is pretty clearly due to its capacity to selectively inhibit serotonin uptake. That's what the medication does, right? I admit that the study on long-lasting placebos was interesting–a year is a long time for a placebo effect, I wouldn't have guessed it. I don't deny the existence of a placebo effect, but neither do I deny the existence of a “medical” effect of these targeted medications. Sometimes they actually work as advertised. And are we willing to extend the placebo effect for five years? Ten? Longer? Because that's how long some people have been successfully taking these drugs by now.

    I worry that an undue emphasis on the placebo effect will help convince people who might benefit greatly from treatment to continue going without it.

  • Anonymous

    Regarding a possible placebo effect of ketamine on depression: We know the average effect size that can be expected from “antidepressant placebos”, an the effect of ketamine is nothing like that. Placebos can't make quadroplegics walk again, and no class of antidepressants has an effect like ketamine. It's like waking the dead. I went through 8 or 9 different medications in 10 years, but ketamine..I realized, I had already forgotten how “normal” feels. Show me one placebo that has an effect that i did not remember any more.

  • reasonsformoving

    Remember, CLPSYCH has something to sell just like everyone else.

  • Neuroskeptic

    Anonymous: I'm not saying that ketamine definitely isn't an antidepressant; I always find subjective reports from experienced people convincing, and it sounds like you've got a lot of experience with various antidepressants. And it's by no means implausible that a drug like ketamine could have a rapid antidepressant effect… after all, ECT does. But the placebo controlled trials to date weren't really placebo controlled, because of active placebo effects (something the authors of these trials acknowledge).

  • Paul

    “I know what alcohol does, not because I've read papers about it, but because I've drunk it. I've also been depressed and taken antidepressants, and for what it's worth, in my experience, some of the drugs currently marketed as antidepressants do have a specific anti-depression effect, although others don't.”

    Ever taken placebo, not knowing it's placebo?

  • Neuroskeptic

    Kind of. I took part in a study comparing an opiate painkiller to placebo. But the design was a bit odd – you got opiate, unblinded, in Session 1, to check if you could tolerate it; then you got opiate or placebo double-blind in 2; then the other one in session 3.

    The effects were pretty strong (dizziness, nausea, euphoria) and I knew what it felt like from Session 1, so I could easily tell that I had opiate in Session 2 and placebo in Session 3. That experience contributed to my scepticism of “double blind” studies that don't include active placebos.

    But I've also taken numerous placebos that I was told were, er, recreational chemicals. And nothing happened. Except the wasted money. Of course strictly speaking I don't know if I've sometimes taken placebos and thought they were real due to the placebo effect, but it seems unlikely. When something works I can tell, I think.

  • Paul

    Very interesting! Do you know of any evidence showing people cannot tell the difference between antipsychotics and placebo? I couldn't find any, which is surprising in and of itself!

  • Neuroskeptic

    I'm not aware of any but I'd be very surprised if anyone couldn't tell they were on (a substantial dose of) an antipsychotic.

    I knew a psychiatrist once who was self-treating with citalopram, but he was in a hurry one day and took a box of chlorpromazine off the shelf (they were right next to each other…) – he said he noticed about half an hour after taking one.

  • Paul

    So the recent review of atypical antipsychotics (leucht et al., 2009) which found they had only a moderate superiority effect over placebo, may simply have been measuring an enhanced placebo response? No included trials tested for blinding, alhough I hear this is no longer recommended.

  • Neuroskeptic

    well it depends on whether the people would be expected to feel better if they knew they were on the drug – if you have someone who's only taking the drug under duress because they don't believe that they're ill, as many psychotic people don't, I'm not sure you would expect an active placebo response.

    But, you would get one if the people rating the symptoms could tell, from sedation probably, what group people were in, and rate the sedated people “improved” when in fact they were just sedated.

  • Paul

    True – but surely the studies recruiting people under duress are few and far between? My guess is, if they exist, they will be the short-term studies (c.24 hours) where drop-out is low and main effect is sedation.

    Most of the studies in that meta-analysis have between 20% and 70% drop-out, over as little as 6-12 weeks. All generally have much greater drop-out from placebo than treatment. This could easily be due to unblinding, rather than treatment efficacy.

    Actually – are you sure placebo won't work for people who don't agree there's a problem? Surely placebo is about expectancies – be they good or bad. People don't want side-effects, but they get them if they are made to expect them…

  • Anonymous

    Maybe this comment will be ignored because this post is old, but I am curious about something. Is it possible to calculate a pooled p value in a meta-analysis ? Because if, as -p- says, removing one study on nine from the Moncrief meta-analysis lower that much the effect size, it seems to me that there is a possibility that you did not considered: that the (pooled) p value is very high, so that the confidence interval on the 0.39 effect size is very wide – leaving open the possibility that the effecct size may be very low (i.e., 0.17) or very high, most likely the first given that 7 of the 9 studies did not find statistically significant effets.
    I would very much like to have your thought on that.
    (On an unrelated subject, did you noticed that Cordelia Fine quote you briefly in her “Delusions of Gender” ?)

  • Anonymous

    For the quote, it is p.167 of the paperback edition.



No brain. No gain.

About Neuroskeptic

Neuroskeptic is a British neuroscientist who takes a skeptical look at his own field, and beyond. His blog offers a look at the latest developments in neuroscience, psychiatry and psychology through a critical lens.


See More

@Neuro_Skeptic on Twitter


Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Collapse bottom bar