Beyond Self-Report

By Neuroskeptic | August 29, 2012 7:04 am

If you want to learn about someone, should you ask them?

Two bits of research published recently cast doubt on the validity of self-report as a tool in psychology and psychiatry. The first found that teens who reported that they suffered from bullying experienced more mild ‘psychotic-like’ symptoms. That correlation would be consistent with the idea that these symptoms arise as a response to stress.

However – the same study found that there was absolutely no correlation between peer ratings of whether someone was bullied, and their psychotic symptoms. Only self-report was associated.

The second study looked at whether a crisis intervention program – intensive mental health care – helped people who’d recently attempted suicide. The results showed that, compared to a control condition, suicidal patients who got crisis care self-reported fewer subsequent suicide attempts.

But a trawl of hospital records painted exactly the opposite picture – the intervention group were more likely to end up in hospital with a second attempt.

Remarkably few studies in psychology and psychiatry compare self-report to other measures of behaviour. This is because self-report is typically a lot easier. If these papers are anything to go by, however, this is a serious limitation. Self-report can be radically at odds with other measures of behaviour.

Which raises the question – who’s right? Are the self-reports right, when they clash with other sources? I don’t think there’s an easy answer. In the bullying case, it might be that self-report is more accurate, because the peers doing the peer-report are the bullies. In the suicide study, maybe the self-report was more accurate, because the patients knew about attempts that never made it to hospital.

But on the other hand you could argue exactly the opposite. Maybe the self-reports of bullying just reflected whether the kids thought their classmates liked them. Maybe the patients were ashamed to admit that they’d reattempted suicide even though they’d got all this special crisis care. It’s hard to tell.

One thing’s clear though: self-report is not the whole story.

ResearchBlogging.orgGromann PM, Goossens FA, Olthof T, Pronk J, and Krabbendam L (2012). Self-perception but not peer reputation of bullying victimization is associated with non-clinical psychotic experiences in adolescents. Psychological medicine, 1-7 PMID: 22895003 

Morthorst B, Krogh J, Erlangsen A, Alberdi F, and Nordentoft M (2012). Effect of assertive outreach after suicide attempt in the AID (assertive intervention for deliberate self harm) trial: randomised controlled trial. BMJ (Clinical research ed.), 345 PMID: 22915730

CATEGORIZED UNDER: mental health, methods, papers, selfreport, you
  • http://www.blogger.com/profile/04330057648612438692 mount analogue

    In this study, self-report is a measure of the individual's perception of bullying, as opposed to some more objective or external measure of bullying. This raises some interesting questions:

    Are self-reports merely exposing the negative bias in individual's perceptions? If so, is this bias driving correlations with other traits, such as neuroticism or psychoticism?

    Even more importantly, can an event that is identical in objective terms be traumatic for one person and not for another? If so, what do we mean we we talk about objective or external measures of experiences such as bullying, or trauma? Aren't they really subjective terms anyway?

    There is a long tradition in cognitive psychology of viewing biased and unrealistic attribution as the one of the primary causes of mental disorder, especially in such as anxiety and phobia.

    For this reason, the actual reality of the situation is important, because we want to know something about the actual environment that is being distorted in the individual's perception. Because so much of CBT works by trying to re-align individual bias with more realistic, objective measures, it would be really useful to map the differences between self-report and other's reports.

    I susect, however, that the main reason self-report is so often used as a measure (beyond the fact that it measures very useful attributions) is that it is very easy and cheap to implement. Investigating the 'real' story is always going to be a lot more difficult.

  • http://www.blogger.com/profile/06647064768789308157 Neuroskeptic

    Exactly right.

    “Can an event that is identical in objective terms be traumatic for one person and not for another?”

    Certainly they can. I knew an ex-police officer once and one of his first jobs after he started was to go and break the news to a woman that her husband had died in an industrial accident.

    So he turned up, dreading how she would react to this horrible trauma.

    But in fact she was glad that he'd died, because she'd hated the guy, and broke open a bottle of wine to celebrate.

  • http://www.blogger.com/profile/03305657400941743430 Russ Poldrack

    Not clear what's new here – concerns about self-report go way back, at least to Nisbett & Wilson in 1980 (http://people.virginia.edu/~tdw/nisbett&wilson.pdf)

  • http://www.blogger.com/profile/16203083806436919715 Bernard Carroll

    As you say, biases can go both ways. In clinical trials of depression treatments, patients’ self reports are routinely disregarded by regulators. Observer ratings trump self ratings of depression severity for judging outcome. Maybe that’s why well over 50% of U.K. patients don’t bother to renew their prescription for an antidepressant drug after 30 days (see the Figure in PubMed ID # 10837883).

    I always prefer to see concordance of self ratings and observer ratings before getting enthusiastic about a claimed new treatment. After all, it’s the patient’s depression!

  • http://www.blogger.com/profile/06647064768789308157 Neuroskeptic

    Russ: You're right it's not new. But I think these two papers provide very striking and topical demonstrations of the issue.

  • http://beyond-advaita.blogspot.com/ ramesam

    Self Report is mostly “the intention” — how the reporter would 'want' the world to be. The fact of how the world actually is, is missed many times by the person reporting.

    I usually rate very low psycholgical studies based merely on questionnaire approach.

  • Anonymous

    The person with self report you would first wonder now after being bullied if they are stuggling with depression or ad/hd. As to a outsider person looking and doing the test.

  • Anonymous

    I wonder if the suicidal patients self-report a lack of symptoms to avoid hospitalization.

    I also wonder if some depressed patients “get better” after electroconvulsive therapy to avoid continuing treatment.

  • http://www.psychologytoday.com/blog/you-illuminated Joshua Gowin

    I think your conclusion is correct: self-report is not the whole story. However, trying to determine if self-report is better or worse than other measures creates a false dichotomy. It certainly provides different information than behavioral measures or peer/therapist ratings. Ultimately, no matter what index you use, the results are subject to noise. In my last lab, studying impulsivity, we often used self-report and behavior in conjunction. One reason is because some populations may have poor introspective ability, limiting the usefulness of self-report. On the other hand, behavioral measures have their own limitations. Using both in combination reduces noise.

  • http://www.blogger.com/profile/09768196006034064120 Unknown

    Perhaps I'm missing the context from withing your field, and you're commenting on undue bias towards using self-report questionnaires, but isn't the issue here more about applying self-report to the wrong sort of question?

    As self-reported studies give you the subjective perception of the situation, obviously this is not going to perfectly match up with objective reality even for normals, let alone suicidal or psychotic individuals.

    If you're asking self-reported questions about subjective subjects, like “how happy/energetic/depressed/hungry are you, 1-10″, and then plotting that over time, then you get a meaningful data series.

  • http://www.blogger.com/profile/06647064768789308157 Neuroskeptic

    Unknown: Yep, I think that's right. Although even with “subjective” measures, self-report is fraught with problems e.g. if you ask someone to rate their “happiness 1-10″, different people may interpret “1″ and “10″ very differently.

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Neuroskeptic

No brain. No gain.

About Neuroskeptic

Neuroskeptic is a British neuroscientist who takes a skeptical look at his own field, and beyond. His blog offers a look at the latest developments in neuroscience, psychiatry and psychology through a critical lens.

ADVERTISEMENT

See More

@Neuro_Skeptic on Twitter

ADVERTISEMENT
Collapse bottom bar
+

Login to your Account

X
E-mail address:
Password:
Remember me
Forgot your password?
No problem. Click here to have it e-mailed to you.

Not Registered Yet?

Register now for FREE. Registration only takes a few minutes to complete. Register now »