Brain Scans: Don’t Throw Out The Baby With The Dead Salmon

By Neuroskeptic | April 11, 2014 2:48 am

Is neuro-skepticism in danger of going too far? Is it time to take a critical look at critiques of neuroscience?

Martha Farah of the University of Pennsylvania says yes, in a Hastings Center Report just published: Brain Images, Babies, and Bathwater: Critiquing Critiques of Functional Neuroimaging

Farah covers a broad spectrum of criticisms, ranging from ‘fMRI doesn’t measure brain activity directly, it only measures blood oxygen’, to the Voodoo Problem, and the charge that ‘brain images exert a seductive allure.’ I don’t have space to discuss these in detail, but I can recommend the paper as a clear and balanced discussion of these tricky issues.

She concludes on a moderately positive note (emphasis mine):

Inferences based on functional brain imaging, whether for basic science or applications, require scrutiny. As we apply such scrutiny, it is important to distinguish between specific criticisms of particular applications or specific studies and wholesale criticisms of the entire enterprise of functional neuroimaging.

In the first category are criticisms aimed at improving the ways in which imaging experiments are designed and the ways in which their results are interpreted. Uncontrolled multiple comparisons, circular analyses and unconstrained reverse inferences are serious problems that undermine the inferences made from brain imaging data. Although the majority of research is not compromised by any of these errors, a substantial minority of published research is, making such criticisms both valid and useful.

In contrast, the more sweeping criticisms of functional imaging concern the method itself and therefore cast doubt on the conclusions of any research carried out with imaging, no matter how well designed and carefully executed. These more wholesale criticisms invoke the hemodynamic nature of the signal being measured, the association of neuroimaging with modular theories of the mind, the statistical nature of brain images, and the color schemes used to make those images seductively alluring.

As mentioned earlier, each of these criticisms contains an element of truth, but overextends that element to mistakenly cast doubt on the validity or utility of functional neuroimaging research as a whole. None of the criticisms reviewed here constitute reasons to reject or even drastically curtail the use of neuroimaging.

Farah’s distinction between ‘particular’ and ‘wholesale’ criticisms of neuroimaging is a very important one. I prefer to use slightly different terminology here, characterizing criticisms as ‘pragmatic’ vs ‘radical’. While pragmatic critics question how we scan brains (and how we can do it better), radical critics question why we do so (and whether we even should.)

Yet these two genres of criticism are often conflated in discussions of neuroskepticism. For example, the famous fMRI scan of a Dead Salmon was created to illustrate a purely pragmatic point: the need for multiple comparisons correction. The authors showed that if you don’t use appropriate statistics, you can ‘find’ neural activity in the brain of a dead fish, whereas if you do it right, the fish stays dead.

salmonThis was an important point, but it was only ever meant to be a technical one. Yet five years later, the Salmon is regularly mentioned in wide-ranging discussions of ‘the limits of fMRI’ or ‘neuroskepticism’. I’ve even seen this narrow statistical example share a page with a philosophical discussion of qualia. The two issues really have nothing in common. I’m afraid to say that nowadays, a good litmus-test for a superficial piece of writing about fMRI is whether it briefly mentions the Salmon before moving on to something else.

Confusing the pragmatic criticisms with the radical ones in this way blunts the impact of both.

But neuroscientists shouldn’t forget that both kinds of criticism can have merit. In general, neuroscientists are good at discussing pragmatic issues. We’re constantly improving neuroimaging methodology and fixing problems with statistics. However, I worry that we are not so willing to give radical criticisms a hearing.

For example, Farah discusses the (radical) argument that neuroimaging doesn’t tell us much because all it tells us is where things happen in the brain, not how they happen. Farah correctly points out that not all imaging studies are founded on a localization (where-in-the-brain) approach. But she notes that some studies are guilty of this. I would add that in some fields of neuroscience (e.g. psychiatry), the proportion is rather high.

So if the critics of localization are right, much of modern neuroscience is barking up the wrong tree. The critics may or may not be right, but I think neuroscientists should be taking a more active role in this debate. We must be willing to confront the possibility that not just our methods, but also our basic assumptions, need to evolve and improve.

ResearchBlogging.orgFarah MJ (2014). Brain images, babies, and bathwater: critiquing critiques of functional neuroimaging. The Hastings Center Report, 44 Suppl 2 PMID: 24634081

  • yokoausjapan

    Hey there. I’ve spent some time thinking about this issue the last weeks and also got the impression that the “where” seems to be of much importance to many scientists. To get a more valid understanding of how the brain works, we might have to address more “w-questions” to the brain. Like “when” (challenging temporal resolution), “what” (taking the sequence of processing steps in different areas into account) or “why” (emphasizing causal relationships, not merely the practice of putting different samples into the scanner and seeing what blinks up). I believe that such questions can be answered via method development and moreover a cautious procedure of setting up an experiment.
    What do you think?

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      I strongly agree!

    • Samuel Lieblich

      Agree too: The involvement of the dead salmon is not a coincidence. Why didn’t they image a dead rat? Because too many fMRI studies are fishing expeditions. Studies should be driven by a strong “Why”.

  • kaleidocyte

    The distinction between pragmatic versus radical critiques is useful in science generally. There are those critics (radical critics) who want us to view the limitations of our field as an impenetrable wall. This is a defeatist perspective from which no knowledge can spring. Most scientists typically view limitations as something that can be overcome with the development of better and more sophisticated methodology. Ultimately, that’s how any scientific field grows and matures. We should listen to the critiques of defeatists, but we need not adopt their defeatist view.

  • http://maximum-entropy-blog.blogspot.com/ Tom Campbell-Ricketts

    On the subject of pragmatic vs radical objections to neuroscientific inferences, yesterday I found a short piece written by Alonzo Fyfe, on Neuroscience and Morality.

    He correctly points out that studies of how people make moral judgments are not the same as studies of what is moral.

    I felt he went too far into what you identify as radical objection, though, when he implied that neuroscience is useless for studying morality, and left a comment to that effect. To me, it’s clear that ‘good’ is defined by a correspondence between the state of the world and the relevant decision criteria in the mind of the person making the judgement. It strikes me that neuroscience might be better positioned than any other discipline to directly measure our decision criteria.

    I just wondered what an actual neuroscientist thinks.

  • Pingback: I’ve Got Your Missing Links Right Here (12 April 2014) – Phenomena: Not Exactly Rocket Science

  • Pingback: I’ve Got Your Missing Links Right Here (12 April 2014) | Gaia Gazette

  • Pingback: Spike activity 11-04-2014 « Mind Hacks

  • Pingback: Spike activity 11-04-2014 | Connecticut Neurofeedback

  • Pingback: Spike activity 11-04-2014 | NYC Neurofeedback

  • Pingback: Check your neuroskepticism | Designolics

  • brainz

    Farah is correct that these criticisms are limited to specific issues, and are therefore insufficient to warrant wholesale dismissal of fMRI as an experimental tool. But there is a more general criticism of fMRI that does warrant blanket skepticism, since it is inherent in all fMRI analytical approaches. This is the assumption that more activity = more involvement. Every fMRI study I know of assumes this in one way or another. Yet there is good reason to believe that this is not true in practice.

    First, if the brain were constantly ramping up activity in response to everyday events (e.g., identifying your coffee mug among others’ mugs in the office kitchen), this would eat up too many resources. Indeed, the old saw about using only 10% of our brains is correct—except it is a different 10% at each moment (really, the percentage of neurons that can fire simultaneously is probably closer to 1% or less; see: Attwell D and Laughlin SB, 2001, An energy budget for signaling in the grey matter of the brain. Journal of Cerebral Blood Flow & Metabolism 21: 1133–1145; Lennie P, 2003, The cost of cortical computation. Currents in Biology 13: 493–497.)

    So increased activity in a given region probably indicates confusion or FAILURE to accomplish the task, rather than success. Since such failures are relatively rare in normal life, metabolism is increased only when something strange happens—for example, when humans are faced with weird tasks (e.g., identify the orientation of a small blob while circles of random colors spin around it) while inside an extremely loud, claustrophobia-inducing tube.

    Moreover, there is evidence that the brain really does show LESS activity when it accomplishes some task, rather than more—even in the scanner. So, for example, when the brain resolves which object a given edge belongs to, activity in primary visual cortex goes down, relative to when the brain has not resolved the object. See: http://www.journalofvision.org/content/8/6/393. In other words, when the brain succeeds, it costs less energy than when the brain fails. Since we perform this kind of object recognition task hundreds of times per day, it makes sense that it should be low cost.

    Now, you could argue that this example itself proves that fMRI is good for something after all, but there are studies that show the opposite pattern of relationship between activity and involvement (i.e., increased activity in primary visual cortex for similar tasks).

    The failure is thus not one of the technology but rather of our theoretical understanding of the brain. fMRI could be useful if we knew more about the fundamentals of the neural code. But since fMRI studies are almost always data-driven, rather than theory-driven (out of necessity since we don’t really have any good fundamental theories about the neural code currently), we should indeed remain highly skeptical about the interpretation of fMRI results.

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      Thanks for an extremely interesting comment – I will probably end up doing a post about this.

    • Ibn al-Haytham

      I strongly support the approach you outlined in the last paragraph, but what do you think about fMRI studies that look for causal connectivity between regions? Those seem to depend on the activity-involvement assumption only to the extent that usually there is an activity-dependent component in the region-of-interest definition procedure.

  • Pingback: Check your neuroskepticism | It's like, Really?

  • Ross Munro

    Can someone clear up a small issue I have always had with brain studies of any sort. It’s a chicken and egg question: what comes first – the thought or the brain activity? In your article I read “For example, Farah discusses the (radical) argument that neuroimaging doesn’t tell us much because all it tells us is where things happen in the brain, not how they happen.” Does this refer to my chicken and egg question? And why is that a radical idea?

  • Pingback: Brain Scans: Don't Throw Out The Baby With The Dead Salmon … | Neurophysiology – What is neurophysiology?

  • Pingback: Check your neuroskepticism | Globe Law News

  • Pingback: Check your neuroskepticism | Blogsfera

  • Pingback: Is neuro-skepticism in danger of going too far?

  • Pingback: ScienceSeeker Editors’ Selections: April 13 – 19, 2014 | ScienceSeeker Blog

  • Pingback: Check your neuroskepticism - Puffin Computers

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Neuroskeptic

No brain. No gain.

About Neuroskeptic

Neuroskeptic is a British neuroscientist who takes a skeptical look at his own field, and beyond. His blog offers a look at the latest developments in neuroscience, psychiatry and psychology through a critical lens.

ADVERTISEMENT

See More

@Neuro_Skeptic on Twitter

ADVERTISEMENT
Collapse bottom bar
+

Login to your Account

X
E-mail address:
Password:
Remember me
Forgot your password?
No problem. Click here to have it e-mailed to you.

Not Registered Yet?

Register now for FREE. Registration only takes a few minutes to complete. Register now »