Words On The Brain: A Semantic Map of the Cortex

By Neuroskeptic | April 30, 2016 11:15 am

In a new Nature paper, Berkely neuroscientists Alexander G. Huth and colleagues present a ‘semantic atlas’ of the human brain. Huth et al. have mapped which brain areas respond to words, according to the semantics (meanings) of each word. It turns out that these maps are highly similar across individuals – which could have implications for ‘mind reading’ technology.

Huth et al. recorded brain activity with fMRI while seven volunteers listened to over two hours of audio narrative (taken from the award winning ‘Moth Radio Hour‘ series.) The purpose of this was to determine the pattern of neural responses associated with different words.

In total, there were over 10,470 English words in the Moth narratives. To make it feasible to analyse these, Huth et al. first calculated the co-occurrence between each word and 985 common English words, which served as semantic ‘markers’. For instance, a word which has a high co-occurrence with “worry” is likely to mean something related to anxiety. Finally, Huth et al. used principle component analysis (PCA) on the co-occurrence data to find a small number of factors or components.

These components are high-level semantic domains. For instance, component 1 (PC1) indexes the ‘humanness’ of words, with high scores on PC1 for words related to categories related to humans and human society and emotions. PC2 seemed to capture the “sensoryness” of words, with concrete, sensory terms scoring high on PC2 while abstract, intangible words scored low.

So where in the brain are these semantic categories encoded? The image below shows the data from one of the participants, S2. Neural activity in response to PC1 is shown in red, PC2 is green and PC3 is blue.

gallant_semantic

The patches in the centre of the image show the cerebral cortex unfolded into a flat sheet. This contains the same information as the more familar 3D brain images, seen at the bottom of the picture. The unfolded sheet is a less intuitive view, but it has the advantage of showing the whole cortex at once.

So are these semantic maps unique to each individual? No – they seem to be largely shared across individuals. Here’s the maps from three of the other participants (S1, S3 and S4). It’s clear just from eyeballing this that, broadly speaking, these maps are very similar, suggesting that we might all have more or less the same semantic atlas:

brain_Semantics

This is an extremely cool study. The results aren’t especially surprising – plenty of previous studies have shown that semantic categories are associated with activity in particular brain areas. But Huth et al. used real-world stimuli to provide an unusually comprehensive approach to the issue.

That said, it’s important to remember that Huth et al. didn’t map the brain responses to individual words. The semantic atlas deals with general word categories. This is relevant to the question of whether Huth et al.’s findings could be used to help “read minds”. As far as I can see, while it might be possible to use Huth et al.’s method to determine whether someone is thinking about (say) ‘child’ or about ‘yellow’ – as these words/concepts fall into very different categories – it would prove much more difficult to distinguish ‘blue’ from ‘yellow’ or ‘son’ from ‘daughter’.

So Huth et al. might be able to read your brain and work out what kind of thoughts you are having, but probably wouldn’t know what those thoughts were, specifically. In other words, I don’t think this paper heralds the arrival of fMRI ‘mind reading’ – not yet, anyway.

ResearchBlogging.orgHuth AG, de Heer WA, Griffiths TL, Theunissen FE, & Gallant JL (2016). Natural speech reveals the semantic maps that tile human cerebral cortex. Nature, 532 (7600), 453-8 PMID: 27121839

CATEGORIZED UNDER: fMRI, papers, select, Top Posts
ADVERTISEMENT
  • lump1

    Please present your data as an old-timey phronology map.

    • http://www.mazepath.com/uncleal/qz4.htm Uncle Al

      Would your “old-timey phronology” be phonology or phrenology? I vote for the drawing of fire from spirits of the points in acupuncture, or maybe rolling naked in a cholla patch and assigning p-values thereafter.

  • Diana

    This is a technologically astonishing work.

    However, I have some doubts that what we see here are “semantic maps”. When looking at the maps, it is obvious that they contradict almost any established fact we know about representation of semantics (e.g. all the stroke literature on semantic deficits). So I tried to think of alternative explanations of the findings.

    “Social” words like “confess”, “guilty”, or “insisted” are associated with pSTS/TPJ. Are these words/concepts represented here? Or do these words often occur in sentences that trigger the participants’ Theory of Mind?
    Words do not only systematically occur in specific sentences but also at certain positions in sentences. A word that systematically occurs in a subordinate clause might correlate with activation of regions associated with chunking. I can imagine similar systematic confounds with other types of syntax processing, attention switching, etc.
    In the many media reports about this study I never read about these alternative explanations. So I am curious to hear what you think about them. Maybe I missed something that rules them out?

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      I agree with you – there is no reason to think that these mapped areas are directly implicated in the “neural representation” of these semantic concepts.

      Rather, they may be activated because they lie “downstream” of the concepts, or are otherwise correlated with them, as you’ve said.

      On the other hand, from a practical point of view (for “mind reading” purposes), this might not matter.

      • Diana

        You’re right – for mind reading purposes that’s fine. But I wonder how they can publish this work without loosing any word about these substantial confounds and the discrepancies with e.g. lesion data. (maybe that’s why it was over two years under review?)
        I guess without the label “semantic” the media echo would be much weaker (and no publication in Nature).

        The authors praise their study design as particularly strong because of its naturalistic stimuli. That’s a cheap argument. The truth is that this is a fundamental weakness of the study that makes the results impossible to interpret.

        • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

          True. The closest they come to discussing this issue is where they write

          “One striking aspect of our atlas is that the distribution of semantically selective areas is relatively symmetrical across the two cerebral hemispheres. This finding is inconsistent with human lesion studies that support the idea that semantic representation is lateralized to the left hemisphere13. However, many fMRI studies of semantic representation find only modest lateralization1 and one study that used narrative stories found highly bilateral results similar to ours2. This suggests that right hemisphere areas may respond more strongly to narrative stimuli than to the words and short phrases used in most studies. Still, more research will be needed to determine what roles these left- and right-hemisphere semantic areas have in language comprehension”

          • http://venpopov.com Ven Popov

            I agree with Diana, while this is indeed a very impressive piece of research, I’m doubtful that their visualizations constitute “semantic maps”, in that most of the cortex identified there is more probably involved in the processing of representation rather that representing things per se. As you say, the differences they found might reflect processing-differences for types of content, rather than representational differences.

            As for the left-right lateralization, I’m surprised that they didn’t have a larger discussion about this, since there’s a large literature that shows right MTG involvement during metaphor processing, sentence completion and making inferences during narratives, suggesting that while the left MTG might be involved in representing single concepts, the right may be responsible for extracting/constructing overall meaning of sentences that is necessary for comprehending larger passages and making inferences. On the other hand, most studies of semantic processing use single words as stimuli (in fact almost all of the studies in Binder et al, 2009 meta-analysis) that show left-lateralization use single word stimuli.

          • Ingid

            Did you see the supplementary material? http://www.nature.com/nature/journal/v532/n7600/extref/nature17637-s1.pdf. They have a lengthy discussion in section 7 how their results relate to the lesion literature. I haven’t digested it fully yet, but looks relevant for your discussion.

    • OWilson

      Apologies for putting some of your ideas in my simple post above.

      I didn’t see yours, and anyway your take was more far more professional :)

  • David Ziegler

    Where is the “BULLSHlT Center” in this concept of brain mapping? Mine is really “active” after reading this.

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      Why so?

  • Anders Eklund

    Nice work, but can you really draw these conclusions from 7 subjects?

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      The maps seem very stable across those 7 people, but with a larger (and, in my view most importantly, diverse) sample we might discover more variability of the maps.

      • Dale Barr

        The consistency across individuals is not entirely surprising, because they are all listening to the same short stories. What *is* surprising, to me at least, is the massive enthusiasm and lack of skepticism in the neuroimaging community toward this paper, which is based on a small sample of restricted language materials (N=10, all from the same radio program) on a very small set of subjects (N=7). These numbers won’t get you through the door at any reputable psycholinguistics journal. Five decades of research in psycholinguistics, starting with Coleman (1964), warns of the dangers of treating language materials as fixed rather than as sampled. This literature is simply ignored, along with a huge neuropsychological literature on the semantic organization of cortex based on lesion studies.

        Much of the consistency seen across the subjects is likely to reflect properties of the materials rather than general lexical semantics. It is not possible to know from this study how well the findings would generalize beyond these stories; at worst, the study might be showing us nothing more than the cortical response to The Moth Radio Hour.

        As impressive as the analysis techniques may be–and maybe their novelty alone warrants publication–we should not allow them to blind us to the huge leap of faith it takes to generalize the results beyond these particular subjects listening to these particular materials. It’s fine to get excited about the method, but wrong to think that this tells us anything general about the semantic organization of cortex.

  • OWilson

    Concepts and emotions mapped, maybe, but words?

    Words are corrupted, twisted and misused all the time so it seems like a fool’s errand.

    Investment
    Fair
    Gay
    Profiling
    Normal
    Liberal
    Prejudice
    Discriminating
    Deviant
    Profit

    All have new (and sometimes opposite) meanings today, and all should trigger different emotions, depending on age and/or political bent.

  • Pingback: Neuroskeptic – | Amer Cavalheiro Hamdan()

  • Paul Scotti

    It appears as if there is activity across the entire cerebral cortex? I
    would have thought that there would be at least some localization for semantic processing.

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      Yeah, it’s most of the temporal lobe and the prefrontal cortex.

  • Pingback: Detailed Map of Language Representation in Human Brain | Neuroscience Institute()

  • Pingback: Semantic atlas opens door to decoding what you want to say | Center for Neural Engineering & Prostheses()

  • Pingback: Virtuality Bites – Enlaces interesantes de la semana – antroposcopio()

  • Pingback: Virtuality Bytes – Intersting links of the week 7 May, 2016 – QUEROLUS.ORG – A DIGITAL LIFE EXPOSED()

  • Pingback: Morsels For The Mind – 06/05/2016 › Six Incredible Things Before Breakfast()

  • Pingback: I’ve Got Your Missing Links Right Here (7 May 2016) – Phenomena: Not Exactly Rocket Science()

  • Adrian Morgan

    As I said when I first heard about this, it sounds like the perfect
    research to volunteer for. Essentially, lying down and listening to a
    story. If the next step is to investigate the word maps of, say, people on the autism spectrum, sign me up.

  • db

    For a psycholinguistic doing research on narrative comprehension, taking fMRI scans of people listening to such natural stimuli equals to those studies scanning spontaneous thoughts about their spouses, or God, or chocolate. Totally lacking in experimental design.

  • Pingback: Science Friday: 3-D Map of Objects in Earth’s Orbit | The Bottom of the North American Continent may be Peeling Off | Words On The Brain – A Semantic Map of the Cortex | sierracynic()

  • Darren K

    Wow, such hate for this research. But not a lot of facts backing up such harsh opinions. I guess there are lot of envious people here who didn’t thought of making themselves the first semantic map of the brain.

  • Ray Black

    It could be that the event of hearing the word collected data might be based on the word heard, and not the word thought.

  • Pingback: Un mapa semántico del cerebro - www.yotumi.com()

  • Pingback: Why do parents call their kids by the wrong names? — Quartz()

  • Pingback: URL()

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Neuroskeptic

No brain. No gain.

About Neuroskeptic

Neuroskeptic is a British neuroscientist who takes a skeptical look at his own field, and beyond. His blog offers a look at the latest developments in neuroscience, psychiatry and psychology through a critical lens.

ADVERTISEMENT

See More

@Neuro_Skeptic on Twitter

ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Collapse bottom bar
+