DSM-5: A Ruse By Any Other Name…

By Neuroskeptic | January 13, 2013 9:45 am

In psychiatry, “a rose is a rose is a rose” as Gertrude Stein put it. That’s according to an editorial in the American Journal of Psychiatry called: The Initial Field Trials of DSM-5: New Blooms and Old Thorns.

Like the authors, I was searching for some petal-based puns to start this piece off, but then I found this “flower with an uncanny resemblance to a MONKEY” which I think does the job quite nicely:

Anyway, the editorial is about the upcoming, controversial fifth revision to the Diagnostic and Statistical Manual (DSM) of the American Psychiatric Association (APA).

A great deal has been written about the DSM-5 over the past few years, as “the rough beast, its hour come round at last / Slouches towards Bethlehem to be born (see, I can reference early-20th-century poetry too).

But now the talk has moved into a new phase, because the results of the DSM-5 ‘field trials’ are finally out. In these studies, the reliability of the new diagnostic criteria for different psychiatric disorders was measured. The new editorial is a summary and discussion of the field trial data.

Two different psychiatrists assessed each patient, and the agreement between their diagnoses was calculated, as the kappa statistic, where 0 indicates no correlation at all and 1 is perfect.

It turns out that the reliabilities of most DSM-5 disorders were not very good. The majority were around 0.5, which is at best mediocre. These included such pillars of psychiatric diagnosis like schizophrenia, bipolar disorder, and alcoholism.

Others were worse. Depression, had a frankly crap kappa of 0.28, and the new ‘Mixed Anxiety-Depressive Disorder’ came in at -0.004 (sic). It was completely meaningless.

The American Journal editorial was written by a group of senior DSM-5 team members. I’m sure they wanted to write a triumphant presentation of their work, but in fact the tone is subdued, even apologetic in places:

As for most new endeavours, the end results are mixed, with both positive and disappointing findings…Experienced clinicians have severe reservations about the proposed research diagnostic scheme for personality disorder…like its predecessors, DSM-5 does not accomplish all that it intended, but it marks continued progress for many patients for whom the benefits of diagnoses and treatment were previously unrealized.

Remember: this is the journal published by the organization responsible for the DSM and even they don’t much like it.

But the real story is even worse. The previous editions of the DSM also conducted field trials. These trials had a system to describe different kappa values: for example, 0.6-0.8 was ‘satisfactory’.

However, the new DSM-5 studies used a different, lower threshold. They simply moved the goalposts, deeming lower kappa values to be good. At one point, they wrote that values of above 0.8 would be ‘miraculous’ and above 0.6 a ’cause for celebration’, yet this wasn’t the view of previous DSM developers.

The indispensable 1boringoldman bloghas a nice graphic showing the results of the DSM-5 trials, with the kappas graded according to the old vs. the new criteria. As you can see, the grass is greener on the new side.

The fact is that the DSM-5 field trial results are worse than the results from DSM-III, the 1980 version that’s served mostly unchanged for 30 years (DSM-IV made fairly modest changes.) The reliabilities have got worse – despite the editorial’s claims of ‘continued progress’. It’s true that the DSM-5 field trials were a lot bigger and conducted rather differently, but still, it’s a serious warning sign.

Finally, there was great variability in the results between different hospitals – in other words the reliability scores were not, themselves, reliable. Some institutions achieved much higher kappa values than others, but it’s anyone’s guess how they managed to do so.

Still, there’s great news: the DSM-5 is just a piece of paper (well, a big stack of them). Any psychiatrist is free to ignore it – as the creator of the more reliable DSM-IV (not III, oops) is now urging them to do.

ResearchBlogging.orgFreedman R, Lewis DA, Michels R, Pine DS, Schultz SK, Tamminga CA, Gabbard GO, Gau SS, Javitt DC, Oquendo MA, Shrout PE, Vieta E, and Yager J (2013). The Initial Field Trials of DSM-5: New Blooms and Old Thorns. The American Journal of Psychiatry, 170 (1), 1-5 PMID: 23288382

CATEGORIZED UNDER: crazylikeus, history, mental health, papers, woo
  • Black Book doc

    Every psychiatrist in the UK has- if my informations are correct – to use the ICDs and not the DSMs.

    Trouble are- to my mind:

    1)I know for sure, that at least some of the ICD11 proposals' makers for psychiatry want to be compatible with the DSM5.

    2) Those APA people are marketing geniuses and -supposing the ICD11's makers would come to their senses- they would thyen need to make an effort in publishing a nice and easy to use material…

    3) In UK, and elsewhere academic leaders of opinions are running for the DSM5:

    From: Dorothy Bishop

    Sent: 11 August 2011 10:49
    To: xxx@xxx.ac.uk
    Re: Misrepresentation of Greenfield’s article

    ///I will send you some peer-reviewed papers when I have some free time, but meanwhile, please see Criterion C in the DSM5 proposed revision, as well as the rationale section, which explains the terminology.///

  • http://petrossa.me/ petrossa.me

    Hardly unexpected. DSM-V is a bit like wikipedia, facts by consensus.

  • http://www.blogger.com/profile/15118040887173718391 deevybee

    Dear Black Book Doc
    Citing the DSM5 re definition of the term 'autistic spectrum disorder' is not the same as “running for the DSM5”
    I have been highly critical of it.

  • Black Book doc


    I am afraid my point was that when somebody in your position as a quintessential talented British academic use the DSM as a reference and as an argument of authority we are in troubles!

  • http://www.blogger.com/profile/17412168482569793996 Eric Charles

    As for most new endeavours, the end results are mixed, with both positive and disappointing findings

    This is a hilarious attempt at obfuscation. They want us to consider the DSMV a “new endeavor”!?! If we just spelled it out, and referred to it as “The DSM, FIFTH EDITION”, they could never get away with that.

  • http://www.blogger.com/profile/11702516173987835066 Zigs

    I am curious how the reliability for MDD could be so poor. Any psychiatrist worth his salt should be able to diagnose MDD.

    I am guessing this has to more with changes in other diagnoses which lead clinicians to choose them over MDD.

    For example, changes to bipolar disorders, mixed anxiety and depression and personality disorders might have inadvertently decreased the reliability for MDD.

  • http://twitter.com/jdottan JT

    Wasn't Frances chairman of the DSM-IV Task Force, not DSM-III? Can't wait for the chairman of the DSM-V inevitably excoriating those making the DSM-VI.

    Perhaps we are also running into the limits of clinical judgment? Dawes, Faust and Meehl had a comment about clinical judgment in Science in 1989 that I think is still relevant, especially with the reliability issues of the DSM-V: http://www.ncbi.nlm.nih.gov/pubmed/2648573

  • http://www.blogger.com/profile/06647064768789308157 Neuroskeptic

    Zigs: Yeah, and also, the 0 kappa for Mixed Anxiety & Depression makes no sense – I can see why it would be low, but used intelligently it should at least be of some use.

    You're right, I suspect, that the broadening and addition of other diagnoses may be part of the problem… I also wonder if patients coming into psych clinics these days are just harder to diagnose (reliably) than back in 1980 – because of the general expansion of psychiatry (especially in children & adolescents) meaning that the severe cases get diagnosed earlier, and aren't available to enter the Field Study pool.

    JT: You're right, that was a typo, thanks.

  • http://dxrevisionwatch.com Suzy Chapman

    Black Book doc said:

    “I know for sure, that at least some of the ICD11 proposals' makers for psychiatry want to be compatible with the DSM5.”

    There is a joint undertaking to strive for harmonization between the two systems.

    The APA participates with WHO in the “International Advisory Group for the Revision of ICD-10 Mental and Behavioural Disorders.” There is also a “DSM-ICD Harmonization Coordination Group.” The International Advisory Group has been meeting since 2007 and has been chaired by Steven Hyman M.D. Dr Hyman ceased membership of the DSM-5 Task Force, last year, and it isn't known who now chairs this group.

    From the Summary Report of the 3rd Meeting of the International Advisory Group for the Revision of ICD-10 Mental and Behavioural Disorders, 11 – 12 March 2008, Geneva:

    “C. ICD-DSM Harmonization Group
    Introduced by Dr. Shekhar Saxena

    “The task of this group is to facilitate the achievement of the highest possible extent of uniformity and harmonization between ICD-11 mental and behavioural disorders and DSM-V disorders and their diagnostic criteria. Dr. Saxena emphasized the genuine desire of both organizations to achieve harmonization of the two systems. He described a variety of specific issues related to differences between the DSM and the ICD-10 that are important areas of discussion by the Harmonization Group.

    “The AG endorsed the following statement intended to guide the WHO representatives in their activities as part of the ICD-DSM Harmonization Group: “WHO and APA should make all attempts to ensure that in their core versions, the category names, glossary descriptions and criteria are identical for ICD and DSM. Adaptations of the ICD should be directly translatable into the core version.””

    In a June 2011 presentation to the International Congress of the Royal College of Psychiatrists, the then APA President, John M. Oldham, MD, MS, referred to “Negotiations in progress to 'harmonize' DSM-5 with ICD-11 and to 'retro-fit' these codes into ICD-10-CM.”

    [ICD-10-CM is the US specific “Clinical Modification” of ICD-10, with a current effective implementation date of October 1, 2014.]

    It does not look, in some cases, as though exact concordance between DSM-5 and ICD-11 category names and disorder descriptions is being proposed.

    For example, DSM-5 is set to replace four of DSM-IV's “Somatoform Disorders” with a new category “Somatic Symptom Disorder.”

    But for ICD-11, the current proposal is to replace six ICD-10 “Somatoform Disorder” categories with a new term “Bodily distress disorder (BDD)” with three, as yet undefined, severities.

    For ICD-11-PHC, the abridged primary care version of ICD-11, the proposals, last year, also included a new disorder section called “Bodily distress disorders” which was said would include DSM-5?s [Complex] Somatic Symptom Disorder (with the caveat that new disorders proposed for ICD-11-PHC may not be retained following the ICD-11 field trials).

    It will be interesting to see which new categories for DSM-5 currently proposed to be mirrored in ICD-11 Beta draft don't make it through to ICD-11 final version following analysis of ICD-11's own field testing.

    Nice monkey flower.

  • Black Book doc


    About Frances:

    ///Les vieillards aiment à donner de bons préceptes pour se consoler de n'être plus en état de donner de mauvais exemples.///La Rochefoucauld

    Old men like to offer good pieces of advice to console themselves for not being able to set bad examples anymore (amateur translation.)

    Stil, worth listening to.

  • Anonymous
  • http://www.blogger.com/profile/06647064768789308157 Neuroskeptic

    Thanks Anonymous. I didn't include all of the links to interesting 1boringoldman posts because that would have been endless!

  • Black Book doc


    ///DSM-V is a bit like wikipedia, facts by consensus.///

    Not at all the DSM aren't facts but speculations (about disorders definitions and criteria)which are politically elected into existence.

    One should never underestimate the attraction and moral sense anesthesia effect of being invited (with expenses paid) to participate in a DSM -or ICD for that matter- comittees, sometimes in attractive places with the top expert of the field…

  • Anonymous

    Fair point. I mentioned those first two posts in particular since they were discussing the same article as in your post. And, you had referenced 1BOM elsewhere in your post. Either way, nice post.

  • Anonymous


No brain. No gain.

About Neuroskeptic

Neuroskeptic is a British neuroscientist who takes a skeptical look at his own field, and beyond. His blog offers a look at the latest developments in neuroscience, psychiatry and psychology through a critical lens.


See More

@Neuro_Skeptic on Twitter


Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Collapse bottom bar