Comments on: The machine that identifies images from brain activity alone

By: Nigel

Nigel — Wed, 25 Feb 2009 08:23:15 +0000

There seem to be papers like this, inferring what someone is looking at from brain imaging data, coming out almost every other day now. The gee-whizzery gets a bit tedious. This stuff may be technically impressive (see how clever we are to get a signal out of this terribly noisy data!), but scientifically it is positively pernicious. These sorts of studies tell us nothing about neural perceptual processing that we did not already know, and, much worse, by pulling off these tricks, which will in fact only work under highly constrained and unnatural viewing conditions, they perpetuate and reinforce the simplistic, widespread, but quite false idea, deeply entrenched in popular consciousness, that visual perception is basically just a matter of opening the eyes and letting the information flow in along the optic nerve for the brain to “process” into visual experience (like turning meat into sausages), and that visual cortex is some sort of inner screen where the scene in front of us is redisplayed for the delectation of the inner eye.
The very fact that such “experiments” are done (and presented as contributions to visual science), let alone the fact that they quickly give rise to fantasies about mind-reading machines, that will also work on memories and fantasies, shows that even smart people who know a bit about the neuroscience of vision are not immune to being misled in this way.
But in fact there is all sorts of evidence that visual perception just does not work like that. It crucially, and centrally, involves all sorts of top-down processes driven not only by the nature of the stimulus, but, even more, by the organism’s current condition and its needs and purposes. Seeing is not like making a video of whatever happens to be in front of our eyes; it has a real biological purpose, it is about actively seeking and finding the information we need (or might need) in the plethora of available visual evidence around us. The relevant processes include not only descending, modulatory signals within the brain (though there is lots and lots of that going on at every stage of visual processing, LGN, visual cortex, and just about everywhere else on the visual pathways), but also directed eye movements, which are absolutely necessary to normal vision, and which, in natural viewing conditions, cause the image on the retina (never mind its momentary reflection in visual cortex) to jiggle about, and to change radically several times per second.
The sorts of fMRI experiments described in the blog post can only be made to work by reducing all this visual activity to a minimum by such things as having the subjects lying motionless within an MRI machine and staring fixedly at decontextualized test pictures that are of no purposive significance whatsoever (that is, the fact that they are looking at a flower instead of a rhinoceros, or whatever, makes absolutely no difference at all to the subject’s plans, purposes, and ongoing behavior). Of course you cannot eliminate the top-down processes altogether – the subject would not be able to actually see without them – but if they are dampened down sufficiently the evidence for them in the data can simply be discarded as irrelevant noise, leaving you with a nice clean, and thoroughly deceptive, demonstration of how perception is just like photography with the visual cortex as the film.

By: Ed Yong

Ed Yong — Thu, 06 Mar 2008 11:51:39 +0000

Sadly, I'm not sure I can - there's a reason that there aren't any physics posts on this blog and that's because anything physics that doesn't have the name "Feynman" or "Davies" on it is as impenetrable to me as it is to you. Any readers have suggestions? Good luck to your husband though - a worthy cause indeed. For his info, I'd highlight this paper which I wrote about here as a really nice example of a very well written paper that still retains academic integrity while being a good read. And, obviously, the Origin of the Species, which is arguably the most important biological text for several centuries and can be read by a schoolkid.

By: Luna_the_cat

Luna_the_cat — Thu, 06 Mar 2008 11:39:47 +0000

This is off-topic, for which I apologise — but I figured, if anyone can help on this, it would be Ed Yong or readers here.
My husband is going to be teaching a graduate seminar on clear writing and communication. For illustration purposes, he needs to dig up a really badly written physics paper. Not bad research — just impenetrable and unclear writing. My field is biology and computing, and I’ve already been able to give him examples of that for biology and medicine, but nearly all physics papers are jargon-filled, unclear and impenetrable as far as I’m concerned, so it’s difficult for me to select one or two.
Sorry about sticking this on an unrelated thread, again.
But would you be able to make any nominations for this?

By: caynazzo

caynazzo — Thu, 06 Mar 2008 03:15:47 +0000

I wrote an article about fMRIs and fear conditioning last year, and consider myself somewhat literate on the subject, but this simply blows my mind. I have to remain skeptical until I look at the primary data. And even then…ha

By: Ed Yong

Ed Yong — Thu, 06 Mar 2008 01:13:18 +0000

Reference now up. And you know, the point about the Matrix did run through my head while I was writing this, but it was 1 in the morning and the part of my head that contains the sci-fi references was asleep and I was sorta counting on people being awesomed by the machine and not noticing. :-p

By: Kevin L.

Kevin L. — Thu, 06 Mar 2008 00:24:57 +0000

Very cool, thanks for sharing! The first thing I thought of wasn't The Matrix. I grant that it's been a long time since I've watched The Matrix, but from what I recall this technology is less Matrix and more Final Fantasy: The Spirits Within. In the latter film, the main character's dreams are recorded for later review. In The Matrix, it was my understanding that the digital readout observed by the operators is a reflection of what is happening in the matrix. True, the crew's mind was plugged into the matrix, but the operators could observe events in the program beyond the crew's virtual perceptions. Um... Anyway, definitely looking forward to seeing how this research progresses.

By: Mike P

Mike P — Wed, 05 Mar 2008 19:24:58 +0000

Cool, thanks! Good post!

By: Ed Yong

Ed Yong — Wed, 05 Mar 2008 19:00:47 +0000

Nature, in the issue that comes out tomorrow. When the paper is published online, it will go into the DOI database and I can stick up a citation using the Research Blogging network.

By: Mike P

Mike P — Wed, 05 Mar 2008 18:24:56 +0000

Where is this published?