Tracking Conscious Perception in Real-Time With fMRI?

By Neuroskeptic | June 9, 2014 4:15 pm

What if it were possible to measure your conscious experience, in real time, using a brain scanner? Neuroscientists Christoph Reichert and colleagues report that they have done just this, using fMRI – although in a limited fashion.

Their research has just been published in Frontiers in Neuroscience: Online tracking of the contents of conscious perception using real-time fMRI

The particular conscious perception that Reichert et al tracked was an example of a bistable visual percept. These are images (or, in this case, a movie) that can be seen in one of two ways. If you look at a bistable image for some time, it seems to ‘switch’ repeatedly between one state and the other – even though the image is unchanged. The best known example is the Necker Cube which can be seen as either pointing ‘left’ or ‘right’:


Because the ‘switching’ of a bistable percept is a purely subjective phenomenon, having nothing to do with objective changes in the image, it’s a useful way to study conscious perception. Previous neuroimaging studies have found differences in brain activation associated with the different states of bistable percepts, but Reichert et al decided to investigate if it were possible to measure these in real time.

The percept they used was a movie in which part of a simple line drawing was shown; in each frame a different part of the drawing was displayed, as if the drawing were moving left and right behind a grey screen with a vertical slit in it. The ‘slit’ is depicted below as an orange box, along with the whole drawing, but note that neither of these things were actually shown. What actually appeared on the screen was just a slice of the drawing, as shown on the right below. This slice was constantly changing (on a loop).


When viewing this movie, Reichert et al say, ones perception spontaneously alternates between ‘two curved lines moving up and down’ and ‘one object moving left and right, behind a screen’. In other words, sometimes the brain ‘fills in’ the whole figure, but sometimes it doesn’t, leaving the isolated lines. The alternations between the two impressions happen unpredictably, with anything from 5 to 50 seconds between each ‘flip’.

So, could brain activity be used to work out which perception someone is seeing at any given time? Yes indeed, say Reichert et al. They say that both the ‘filled-in’ and ‘isolated’ versions of the percept are associated with different patterns of neural activity and that, based on these characteristic patterns, the ‘active’ percept can be detected, in ‘real time’ (i.e. with about 10 seconds lag, due to the way fMRI works) with 75-80% accuracy. Just flipping a coin would get about 50% right so this is substantially better than chance.

The design of the study was somewhat complex with several different comparisons but from what I can see, the methods were appropriate.

Perhaps the most striking finding was that the real-time algorithm generalized across individuals (in a leave-one-subject-out validation). So it would be possible to put someone in the scanner, someone who’d not been scanned before, and tell what they were perceiving with no need for calibration. This is pretty impressive, and quite rare for a ‘mind reading’ fMRI algorithm. However, it’s important to remember that this study was only about one very specific aspect of perception.

It’s really pretty amazing when you stop to think about it. Previously, only one person in all the world could possibly know which percept I was experiencing – me. I could tell you about my experience, but as it remained my experience, you’d have to take my word for it. You’d have no way to check.

Now, thanks to Reichert et al, you do have a way. The veil of ignorance between I and thou is not, quite, impenetrable after all.

ResearchBlogging.orgReichert C, Fendrich R, Bernarding J, Tempelmann C, Hinrichs H, & Rieger JW (2014). Online tracking of the contents of conscious perception using real-time fMRI. Frontiers in neuroscience, 8 PMID: 24904260

  • Longmire

    So how long will it take for “science” to prove ESP is real?

    • Bronwyn (デイ)

      Silly, science doesn’t prove anything. It can only support or argue against a given conclusion.

      But ESP is touchy business for which we need finer tools yet, I’d argue.

      • Longmire

        Its just a matter of time though, this technology will readily demonstrate that “minds” synchronize with other “minds” and perhaps give an acceptable explanation for ESP, and certainly change our understanding of the “subconscious”.

        • daqu

          Yes! Or not.

  • Mark Ettinger

    The result is indeed amazing. However it seems to me the impenetrability of experience remains intact. What this experiment may provide is a technology for _predicting_ (a report of) experience but prediction is very different from the experience itself. If you use the fMRI to predict my bistable percept and I declare you incorrect, you have no way to tell if I’m lying or an exception to the regularity. Ultimately, this is not unlike my ability to predict your report of colors when I point to objects without ever knowing if my blue is “your blue.”

    • Neuroskeptic

      It’s true that fMRI doesn’t allow me to experience your experience – and it will never let me know if ‘your blue is my blue’.

      However, before fMRI, I would have absolutely no way of knowing if you were lying or telling the truth if you gave me a report of your bistable percept.

      Now thanks to this paper I can make an informed guess. It might be wrong but it’s most likely correct, and that ability to get some – imperfect – information about your experience (though not the experience itself) is something new, I think…

      …although on the other hand, you could say that I already have this ability in other cases, just not in bistable perception cases. For instance if you are sweating, trembling etc. I can infer that you are experiencing a strong emotion while if you’re sitting calmly, you are probably not.

  • Pingback: Tracking Conscious Perception in Real-Time With...()

  • Bronwyn (デイ)

    Cool, but I feel like they’re really just using slower, more expensive methods to demonstrate what’s already been demonstrated– or less. I found Doesburg et al.’s work from 2009 much more exciting. They also used EEG instead, which removes the vast majority of the time lag:

  • stephanie hancock

    fMRI is pretty awesome when you are trying to see “inside the brain”. I think its awesome that you can use this device to see what people are thinking about at that very moment. I don’t believe however that this device can tell someone if they are lying because you need a visual aid to look at before the device tells you which thing you are thinking about.

  • Jessica Groenke

    This experiment is fascinating. However, it seems that using fmri one would only be able to make a prediction of an experience. Because it is only a prediction rather than the sharing the actual experience one could not tell if I was truly lying or just outside the norm. Using fMRI is indeed an advancement in the right direction, however, I believe what would be more impressive is discovering a way to home in on the activities of the individual neuron, which are vital to mental function.

  • Pingback: Leibniz Stepped into the Mill and Saw a Thought | David J. Frost()

  • Pingback: Leibniz Stepped into a Mill and Saw a Thought, Continued | David J. Frost()

  • Pingback: Tracking Conscious Perception in Real-Time With fMRI? | The Historyscoper's Science and Technology Watch Blog()



No brain. No gain.

About Neuroskeptic

Neuroskeptic is a British neuroscientist who takes a skeptical look at his own field, and beyond. His blog offers a look at the latest developments in neuroscience, psychiatry and psychology through a critical lens.


See More

@Neuro_Skeptic on Twitter


Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Collapse bottom bar