Comments on: 'Talking face' simulations in the brain help us work out what's being said

By: Ed Yong

Ed Yong — Tue, 22 Apr 2008 01:56:30 +0000

I’m convinced that teleconferences are a sick prank played by more senior people than me, for precisely those reasons.
TE, not sure if this satisfies your objections, but the authors did mention the possibility that the subjects were just paying greater attention to the voices that came with a matching video. They claim that their data rules this out. If the benefits were due to attention rather than some property of the faces themselves, then you would expect both controls and prosopagnosics to do better in both speech recognition and speaker recognition tasks. That wasn’t the case – the prosopagnosics gained no advantage in the speaker recognition task.

By: HP

HP — Tue, 22 Apr 2008 01:09:58 +0000

I’ve noticed this a lot when participating in teleconferences, particularly when the participants are scattered across two or three continents, and some of them are not native English speakers. It’s much easier to understand people whom I’ve met at some point than those who are only voices on the phone. So a non-native speaker of English whom I’ve actually met is easier to understand than the native speaker who I only know as a voice.
(I once sat in a weekly meeting where the project manager was a Hindi speaker, who spoke extraordinarily rapidly just as a personal ideolect, and the code developers were Quebecois French speakers, and I’m a midwestern anglo-USian. Most of the meeting consisted of “ImsorryIcouldnotmakeoutthatlastbitplease,” and “Ah don’ unnerstan’ what you say, eh?” This went on forever. But I have no problem understanding either Hindi or Quebec French speakers when we’re all in the same room.)
To follow on TE’s comment, I’m not sure if the study conclusively shows that this is a facial-image simulation in the brain or not, but the phenomenon of having trouble following purely audio cues in ordinary speech is real. As a possible data point, I listen to a lot of old radio dramas. And they’re generally easy (and enjoyable) to follow. But if you break them down and look at the dialog objectively, it’s all highly artificial, and designed to be easily understood without visual cues. Regular folks talking on the phone, especially when dealing with language, dialect, and ideolect — not so much.

By: TE

TE — Mon, 21 Apr 2008 23:54:17 +0000

It runs a simulation of the speaker’s face to fill in any information missing from the sound stream alone.
Maybe I’ve read too much BF Skinner, but I suspect all these results are better explained in terms of attention and learning, and that the activity seen on the MRIs is better described as a correlate of recognition than a cause.
People may generally process the nuances of speakers they’ve previously seen in person much better than those they’ve never seen, but this may only reflect a greater ability to learn from actual speakers than from disembodied voices. Ultimately, the distinctions learned can still be strictly auditory.
People may also show greater tendency to visualize the speakers they’ve seen and comprehend well. But it doesn’t follow that the visualization is integral to the comprehension. And when you really think about it, how could it be?–The visibility of an actual speaker supplies me with additional stimulation relevant to what he or she is saying. My own visualization of the speaker cannot do the same. I can only imagine the speaker accurately if I hear him or her correctly in the first place, in which case the mental simulation would be superfluous.