Aural illusion

By Phil Plait | June 20, 2008 2:28 pm

This is totally amazing.

First, watch this video. It’s short — just a guy saying a single word — and try to figure out what he’s actually saying.

OK, now the cool part: listen to the video again and close your eyes, or look away from the video. Seriously. Do it.

Isn’t that awesome? I was blown away from this. I need to look into it more; I’ve never run into this before. I love illusions, and this must tie in somehow with aural pareidolia.

Tip o’ the lobe to Way of the Woo.

CATEGORIZED UNDER: Cool stuff, Pareidolia, Science, Skepticism

Comments (143)

  1. Huh… no matter how I try it, I only hear “bah bah, bah bah, bah bah”.

  2. Yojimbo

    Ummmm… I don’t get it. I tried it a half dozen times with eyes open and closed – I hear exactly the same thing either way.

    What do you hear? It reminds me of the ambiguous sound the Japanese use for our R, L and D.

  3. John Phillips, FCD

    Irrespective of whether I am listening with my eyes open or closed it flicks back and forth between a ‘b’ and a ‘d’ sound.

  4. AJ

    Possible spoiler?

    Watching him I heard “Da Da”
    Not watching I heard “Ba Ba”

  5. SteveStar

    I hear “Ga Ga” then “Tha Tha”. Not quite sure what the significants of that is.

  6. Joe Meils

    (shrug) I guess I must be dim. It sounds the same to me either way.

    Baa baa baa

  7. Marco

    Oh man! That is awesome!

  8. astronowanabe

    I agree with AJ
    quite distinctly *See* “DaDa” but when just hearing “BaBa”

  9. Dan D

    Ate first i heard “Gaa gaa” with my eyes open, and then “baa baa” with them closed. But after listening and watching it a bunch of times i just kinda hear “aaa aaa” no mater what now. Weird.

  10. Perosha

    I’m not sure I get it. When I watch it I hear “ba ba” even though he’s mouthing what looks like “ga ga,” and when I listen to it I obviously just hear “ba ba.” Hmm…Maybe bad kung-fu movie dubbing has spoiled aural illusions for me? *lol*

  11. Sili

    I’d like to see LanguageLog take this on. Might be able to learn something about phonetics from it.

    Part the issue is obviously our instinctive knowledge that /b/ is a bilabial plosive and when we don’t see that we try to reinterpret the place of articulation as being elsewhere.

  12. DaveR

    I here Ba Ba both ways. Maybe my brain is broken. :)

  13. Michael Campbell

    When I watch, I hear “da da”, when I don’t watch, it’s “ba ba”. The difference is not huge, but it’s definitely there. Even closing my eyes halfway through makes a difference.

  14. dannysalamander

    it seems like he overdubbed audio over the video, if you watch it enough, it starts to seem a little awkward. maybe that’s just me…

  15. Michelle

    …gagah? What?

  16. I’m in the “See Dah-dah but hear Baa-Baa” crowd.

  17. hambr

    Yeah, I’m seeing ga ga but hearing ba ba

  18. Mena

    Ok, I got “tha tha, lah dah, tha tha” with my eyes open, “Bah bah, bah bah, bah bah” with my eyes closed. Is anyone else here multilingual? I’m wondering if that’s why I heard different things with my eyes open and if that makes a difference.

  19. Justin Fowler

    I actually study cognitive psychology and I remember learning about that illusion, if you will, in my undergrad cognitive psych class. It has to do with the visual cues that we use to understand language. A lot of people never realize how much our visual system plays in auditory processes. The fact that our brain actually processes something completely different when a visual cue is added is interesting indeed. If I remember correctly, that phenomenon is a shortcut that our brain uses to quickly process the sound. Our brains need to utilize shortcuts all the time due to the ridiculous amount of information that it takes in at all times. We would be completely overwhelmed if we had to go through every bit of information without finding shortcuts to process it.

    I can not for the life of me remember what it was called when it was introduced to us, but I do remember being very intrigued that day in class. Point in case though, our brains are awesome!

  20. T.phillips

    I don’t get it. It sounds the same either way to me..

  21. Observer

    “I can not for the life of me remember what it was called when it was introduced to us”

    It’s the “McGurk effect”:

    http://en.wikipedia.org/wiki/McGurk_effect

    The video is “borrowed” from Arnt Maasø’s site:

    http://www.media.uio.no/personer/arntm/McGurk_english.html

    which states:

    <>

    and also points out that

    <>

    so if you cannot hear the difference, you either belong to the 2% minority, or you need a better computer.

  22. I didn’t get it either — both experiences sounded the same.

    I think part of the problem was Phil’s statement “try to figure out what he’s actually saying,” leading me to expect a “hidden word” to suddenly pop out the 2nd time.

  23. Chas.

    It’s just “bah-bah” not matter what for me. Visual cues play very little part in *my* understanding of spoken language…unless I am reading lips. Aural cues are principal for me and I am not often thrown by visual distractions.

    I do find it very interesting that other folks might be severely affected by this “cognitive dissonance.” It seems especially important in this campaign season.

  24. Mike

    If you watch the next video, it explains the McGurk Effect. Basically he spliced a video of him saying Gaa Gaa with an audio track of Baa Baa. Our eyes make us think we are hearing Gaa Gaa or Daa Daa due to visual cues, but our ears alone hear the real audio.

  25. Observer

    And I need to learn that WP uses brain-dead sanitizing rules; let’s see if this works better:

    which states:

    Most adults (98%) think they are hearing “DA” – a so called “fused respons” – where the “D” is a result of an audio-visual illusion. In reality you are hearing the sound “BA”, while you are seing the lip movements “GA”.

    and also points out that

    IF you are listening through poor speakers, the illusion might not work. Try connecting external headphones. The effect will also not work if the images are not streaming right, or in other words not in synch with the sound.

    so if you cannot hear the difference, you either belong to the 2% minority, or you need a better computer.

  26. «bønez_brigade»

    Phil, you must also be reading (or have read) ‘Mind Hacks’, no? This video & effect was explained in Hack #59 – “Hearing With Your Eyes: The McGurk Effect”.

  27. Bruce Almighty

    I’ve got it…it’s both, depending on whether you “measure” it by watching or just listening! Quantum sound (QuantroPhonic?) – coming soon to a theater near you.

  28. swede

    I hear “dah-dah” when watching, and “bah-bah” when not.

    My guess is that we interpret it as “dah-dah” when we look at him, because we think it is unlikely that he says “bah-bah” with his mouth still open.

  29. Anchor

    There’s nothing remotely new or interesting, let alone AWESOME in this.

    Anyone who is compelled by nature or habit to listening to sounds very carefully (such as many musicians do to music, for example) are already aware of the similarity between certain sounds. What’s more, many people who have such a habit often find it extremely difficult to make any sense out of vocalized words in music – they can’t tell you at all what a singer in a song “says”, only that a vocalist is present. These people are also most often prone to requiring a speaker to repeat herself to obtain what was said.

    I have long suspected that people may be divided into two groups (although there may be others): those who are verbal-language dominant who can relatively easily assimilate speech, and those who take all sound more or less “raw”.

    Which is best? One might think that having a “processing filter” to immediately grasp speech is a positive selection factor. On the other hand, such an “illusion” demonstrates that those who aren’t dominated by such a filter aren’t as easily fooled by whatever they hear.

  30. Tim G

    Thanks Observer

    I’m with a few others on this:

    Looking, I thought I heard “dah dah…”.

    Not looking, I thought I heard “bah bah…”.

  31. Tim G

    Oops. I meant to link to Observer’s original comment.

  32. Hmm, I must be a freak. When I listen AND watch the video, I hear “La La La La”. When I have my eyes closed, I hear the “Ba Ba Ba Ba”.

  33. Tim G

    Play the video with the sound off. He is indeed saying “gah gah … “.

  34. Daffy

    Absolutely amazing…and confirms what I have always suspected: the human brain is not to be trusted.

    Or, as the Firesign Theatre put it: “Your brain may no longer be the boss.”

    Very cool.

  35. hsj

    Brian Dunning has some aural “illusions” in his latest podcast on http://www.skeptoid.com, they’re pretty awesome. A nice mix of audio and pareidolia.

  36. Daffy

    Odd side note: the curious sense of superiority from those who claim not to see/hear the illusion.

  37. decius

    Glad to be in the 2% minority.

  38. decius

    Not superiority, Daffy, rather gratefulness to nature, being an opera fan.

  39. Observer

    “Glad to be in the 2% minority.”

    Or maybe you just have a crappy computer, or possibly even hearing or vision issues? Without participating in a controlled study, how can you tell?

  40. I hear the same thing both ways, but what I hear is “thaa, thaa.” Not sure if that has anything to do with the illusion.

  41. That was pretty weird…dah dah watching and bah bah listening…a real “huh?” experience.

  42. Daffy

    decius,

    I am a professional musician, not just a fan. I enjoyed the effect immensely.

  43. rob

    when i watch with my eyes open, i hear “Daa Daa.”

    when i close my eyes i hear “Paul is dead.”

  44. Drama

    Wow! That is so freaky. Whenever I look, it is clearly and distinctly “da da da” but when I close my eyes it’s “ba ba ba,” this is really freaking me out, ha! Thanks for posting this, I never realized sound was so… subjective

  45. MarshallDog

    Okay, so I watched this a dozen times before reading the comments and couldn’t figure out what the crap was supposed to be so amazing about it. After reading about what I am supposed to see/hear, now I get it. I do the same thing with those Magic Eye pictures. I don’t see anything until someone tells me what to see. Is my brain messed up?

  46. PhantomPhoton

    I really can’t tell the difference. That said I can’t really tell what he’s saying in any situation. I can’t make out the consonant sounds.

    However I’ve always had an audio language “comprehension” problem. I often find myself not able to understand sung lyrics on the radio. It takes me far longer to comprehend foreign language when it is spoken versus written. I always find much more enjoyment watching movies or television with cc on because I always pick up much more of what’s going on.

    Now another phenomenon I’ve noticed with myself was in school I always had better retention of a subject when I had a teacher who did spoken lectures in front of the class versus other teaching styles like book work or printed notes. But I suspect that had more to do with interaction versus audio/ language learning skills.

  47. brad hala

    sight- the,the…aural- ba,ba
    BA, I gots ta know; what do you hear?

  48. RawheaD

    Funny coincidence. I just came across this blog post:

    http://www.bbc.co.uk/science/humanbody/body/interactives/senseschallenge/

    today and one of the illusions in the challenge deals with this very phenomenon.

    *Note: if you happen to be using one of the most advanced web browsers like Safari (or better yet, the latest nightly of WebKit, which is the only browser that will pass the Acid3 test (http://acid3.acidtests.org/)), it’s going to say you have an outdated browser. If you have the debug menu turned on, you can simply change your useragent to something like IE7 and the flash will operate fine ;p

  49. Geoff

    It didn’t work for me either but I’m a film editor and am used to lip sync issues so maybe that’s it.

  50. Moose

    I was hearing a sound that was half-way between Da Da and Ga Ga, but not quite either. This was equally true with my eyes open or closed.

  51. jokermage

    I’m in the “baa baa” both ways group. Didn’t hear any difference between the two.

  52. Odd. I head Da-da da-da da-da with my eyes open or closed. Although it sounds like a da with a lisp.

    John B. Sandlin

  53. Masterpo

    http://www.youtube.com/watch?v=eQoYKuNcCpU

    This video truly showcase the McGurk Effect in a very effective way .

  54. claschxtreme

    wow,

    bah bah – dah dah – bah dah

    awesome, the first time i watched the video with no sound ( forgot to turn on the speakers) and i couldn’t figure out what he was saying by watching his lips, not that i usually can but it gives an idea anyways.

  55. Adam

    It was “bah bah” for me both ways. The first time I watched with eyes open too….

    I used to do a lot of video editing which entailed some lip syncing audio from other sources from time to time so that may be why it sounds the same to me.

  56. OK. Odd. If I don’t watch it for a while, and concentrate on the fact he’s saying Bah-bah and listen with my eyes closed, I can just make it out that way. But if I watch and hear, I hear the Dah-Dah, and then immediately listen but not watch, I still hear the Dah-Dah. Maybe it’s a persistence of mental image… or something.

    John B. Sandlin

  57. Jose

    I can’t hear a difference with this video, but here’s a link to an episode of Scientific American Frontiers that works for me http://www.pbs.org/saf/1502/video/watchonline.htm

    Play the third video called “Cars That Think Part 3″, and skip to about
    2/3 of the way through. I think it’s the soothing presence of Alan Alda (The Alda
    effect)that makes all the difference.

  58. i haven’t read the other comments.

    from what i can tell, he probably has a sound of him saying baba over a video of him saying dada.

    the reason we hear dada when we’re looking at him is because his lips don’t come together to make the ‘b’ sound, so our mind interprets it as a ‘d’.

    not really mind-boggling, but interesting.

    isn’t that how ventriloquists work? don’t they substitute sounds without their lips giving cues as what they’re really saying?

  59. KC

    Hmm . . . I hear “Bah bah, bah bah, bah bah” both while watching the video and while listening to the words and not watching the mouth.

    Question: How many others who heard no difference also have some musical ability? Is there perhaps a connection?

  60. Brian

    I hear “da da” or “tha tha” watching the video, but “ba ba” when I’m not looking, so I guess I’m one of the “98%”. I think that’s a pretty cool effect. I also am finding it interesting that way more than 2% of the responses here are people claiming that they don’t hear a difference, so I wonder what that says about the people that read sites like this, and how their brains work. Hmmmmm….

  61. Thanks for posting this Phil. I first studied the McGurk effect well before the advent of the Web. Now the Web is the ideal venue for a lot of these cool cognitive phenomena.

  62. Helioprogenus

    I hear a ga ga with eyes open, and a ba ba with them closed. Aren’t these illusions mind blowing? I wonder if one closes their eyes at a ventriloquist speaking, will they also be equally McGurked?

  63. Pop

    What with my bad hearing and clouded eyes, I think he was speaking two languages. I heard Da (Russian for yes) and Bah (English English for an unacceptable situation as in Bah-Humbug). It was the same eyes open or closed.

    He might have been trying to spell something in Polish. W00tz

  64. Radwaste

    I don’t find this anywhere near as entertaining as watching a politician try to claim they never said something that is easily found on YouTube.

  65. Ian

    Huh. It sounded like “tha tha” to me, both ways. I had to listen several times before I started to be able to make out “ba ba” with my eyes closed. I’m still not totally certain whether I’m actually hearing the difference or I’ve just convinced myself that I should be hearing it, though.

  66. BoC

    Amazing! When I watch I hear Raymond Luxury-Yacht, but when I listen with eyes closed I hear Throat-Warbler Mangrove!

  67. Stuart

    Initially, I heard “la la” when watching and still “la la” when not.

    Two hours later (to ensure a cache flush :-) , I decided to try it the other way round – not watching I heard “ba ba”, then watched and heard “la la”.

    Some sort of persistence effect initially?

  68. Geoff

    Wait… I’ve changed my mind. He’s saying “What’s the frequency Kenneth?”

  69. Larry

    Phil,
    You have been listening to too many aliens as you fly around the universe.

  70. Stuart

    Followup to my previous post:-

    Watching I definitely always hear “la la”. Not watching, I now find I can decide in advance whether I’m going to hear “la”, “ba” or “da”, and my brain pretty much co-operates. So I have some control over my internal reality anyway.

  71. BJN

    I suppose that if you look at his mouth and see that he’s not using his lips your visual input is supposed to make make the ambiguous sound into a “D” consonant. It doesn’t sound like a “D” to me, nor a “B” but some kind of muddled, gutteral “B,D,T” -ish sound. It’s the same if I look at the face or not. Guess my visual processor and aural processor don’t network so well.

    Oh well…

  72. Steve

    One time my wife’s family was sitting around the table having a conversation when her aunt said, “Just a minute, let me get my glasses. I can’t hear you.” :)

    This is along the same lines, I suspect – if you watch the mouth, you get a visual clue. As stated, you see the mouth open, so expect “da da”, but with eyes closed, I hear “ba ba”.

    Interesting that some hear it and some don’t.

  73. Quiet Desperation

    Even having read what others heard, it worked. Sweet.

    One time my wife’s family was sitting around the table having a conversation when her aunt said, “Just a minute, let me get my glasses. I can’t hear you.”

    When we moved from New Jersey to California when I was a kid, we made a vacation out of it and drove across the country. Somewhere in New Mexico, my sister (19 at the time) was looking out at the vast wilderness and said, “You ever notice how there’s always a valley between the mountain ranges?”

    We all just looked at her as she tried to better articulate her point. To this day she can’t really explain it.

  74. PG

    Wow. Watching, I hear “tha tha”. Closing my eyes, I hear “ba ba”. Freaky deaky.

  75. Thomas Siefert

    With my eyes open he sounds like slightly inarticulate, but not overtly so considering today’s youth fed up on American Idol.
    With my eyes closed he sounds like BA’s biggest fan finding consolation in his mantra.

    Can’t help thinking that the German band Trio might have had a deeper meaning with their old hit: Da Da Da

    http://youtube.com/watch?v=QDXaW8lfjKk&feature=related

  76. Geoff (on 20 Jun 2008 at 5:02 pm)
    “It didn’t work for me either but I’m a film editor and am used to lip sync issues so maybe that’s it.”

    My situation exactly. I edit video (and it’s audio) professionally and heard “Bah, Bah.” each time and saw his lips saying “Da, Da.” or something. The “B” sound starts with the lips closed (“P” does too). So it looked weird to me from the get-go. He might not even be actually making any sound at all.

    This puts Geoff & me in the 2% category. I’m subconsciously looking for lip sync errors I suppose… Working in broadcast TV does that stuff to ya, it’s no problem, really. I don’t watch TV, I study it. The Wikipedia page says the effect is “robust” but it doesn’t work for me at all. It’s the professional in me always seeing with the unblinking eye and hearing with a critical ear. I don’t miss much. It’s hard to pull the wool over an unblinking eye, you might say…
    Rich

  77. themadlolscientist

    Bob bob bob, bob Bobberann……..

    “Just a minute, let me get my glasses. I can’t hear you.”

    That’s not so freaky as it sounds. People who have partial hearing losses can make up for it to some degree by lip-reading, often without realizing it. When we put on our glasses, we interpret it as being able to “hear” better because we can pick up on visual cues that fill in for the missing sounds (typically sibilants, because we lose the high frequencies first). I have about a 50% hearing loss in my left ear, and I didn’t realize I was compensating until I got my first pair of glasses.

  78. David Marasco

    Yes, your brain does all kinds of processing on sound. One of the more famous cases is the “missing fundamental”. If you play a song that is heavy on the bass on your itty-bitty car speakers, well, those speakers aren’t going to be able to output the low frequency fundamental. However, our brains hear the overtones, and fill in the missing frequency.

    Scary.

  79. Dave Hall

    Watching: Tha tha tha tha
    looking away: Ba ba ba ba

    Then I noticed that guy looks like the Bad Asrtonomer in a Jebus costume!

  80. NZMonkey

    Well, I hear and see the same both times. I guess it’s because I need better specs? :)

    But effect this would go someway to explaining why ads and movies dubbed into English (especially ads that are re-dubbed with local actor’s voices) are so grating?

  81. Nick

    Not on topic, http://www.vidmax.com/index.php/videos/view/7635 may noy be the same but ill be damned if its not funny

  82. Bill Bones

    Watch or no watch, he says Ba-Ba, Pa-Pa, Ba-Ba. Of course, I am not a native English speaker and so it just sounds weird no matter if I see it or don’t… :(

  83. madge

    I am of the Gaa Gaa/Baa Baa persuasion but surely this is a trick that ventriloquists have been exploiting for centuries (as in”a Gottle of Geer” for a “Bottle of Beer”.)

    Oh and Good Solstice to one and all! :)

  84. Gaa Gaa/Baa Baa. Had to liten to it several times, and then looked on YouTube where they posted what the difference was, and only then could I hear the difference.

    Slightly off topic, but is this related to the whole reverse speech woo woo? I’ve heard it, and I’ll be dam*ed if there’s not something to it. What is it? Or is it fraudulent?

  85. Gebo

    The McGurk effect. Hmmm… Interesting.
    You’re sure it’s not the McBurk effect?

  86. alfaniner

    I have found that with very little editing some aural sequences fit pretty well over a completely different video.

    “Darth Vader is not a whiny little kid!”
    <a href=”http://www.youtube.com/watch?v=A38NUMUVowA”

    Numa Numa Ooompa Loompa
    <a href=”http://www.youtube.com/watch?v=w3ggu2WxUUM”

  87. alfaniner

    (really need a preview function… try again)
    <a href=”http://www.youtube.com/watch?v=A38NUMUVowA”
    <a href=”http://www.youtube.com/watch?v=w3ggu2WxUUM”

  88. Tom Marking

    Awesome, nah. If you want to see some awesome aural illusion check this one out. It’s Carl Sagan’s voice from Cosmos superimposed on Agent Smith of the Matrix. This one is hilarious.

    http://video.google.com/videosearch?q=Carl+Sagan+the+Matrix+Agent+Smith&sitesearch=#

    BTW, I see Dah, dah and hear Bah, bah just like 98 percent of people do.

  89. Rav Winston

    Actually, I am hearing a voice telling me to have buttermilk pancakes with Nutella spread on them.

    Wierd!

  90. Josephine

    Seems like as if my brain has not understood that it can make its life easier if it incorporates visual cues. Both when I watch and when I listen (without watching) I hear “ba ba ba.”

    I must switch computers and see if this still is the case. This was an interesting phenomenon of which I was not previously aware. Thank you, BA for telling us!

  91. bubba

    I heard him tell me to rub motor oil on my body. When I close my eyes he tells me to break Charlie Manson out of prison.

  92. bubba

    I’m with the ga-ga/ba-ba team.

  93. OtherRob

    Guess I’m one of the unwashed masses. I heard “Da Da” with my eyes open and “Ba Ba” with them closed. Not earth-shattering or anything, but kinda cool.

  94. Dan J

    I heard BaBa both times. So… am I a mutant or somesuch?

  95. Helioprogenus

    @John Kennel. The big bang depends on your temporal perspective I suppose. If you look at it as time’s arrow moves back, instead of forward, it would definitely look like a contractive event. Apparently however, thanks to recent research, and even articles in Sci Am, perhaps it’s also possible to view the big bang in our temporal perspective as well. As the universe becomes progressively expansive and cold, perhaps hundreds of trillions of years later, a quantum fluctuation in a specific area, can cause a new universe to suddenly collapse out of it, and if successful, an inflationary event will proceed, as time in this space, can move in either direction (from our perspective, which supposedly occurs half the time, we see it moving one way, it would be the reverse in half the other created universes). Anyway, still unclear about many things here, but hope that helps push the discussion you attempted to start.

  96. I heard da da watching it and ba ba with my eyes closed. However, my hearing is somewhat impaired, and anyone who knows me knows I lipread to some degree.

    That makes me particularly susceptible to this sort of illusion.

  97. Santiago

    Bah, this is tame, if you REALLY want your socks blown off, get some stereo headphones (or, you know, any set of headphones, who the hell has mono headphones?) close your eyes and listen to this:

    http://www.youtube.com/watch?v=IUDTlvagjJA

    It’s called the virtual barbershop, and I dare everyone to listen completely through it without opening your eyes.

  98. SteveN

    I’m not in the 2%. I’ve been hearing impaired for many years. I can actually “hear” it three different ways: 1) sound off – ga ga; 2) sound + video, a Japanese sounding – dtha dtha; 3) sound only – ba ba. The video might make a good screening test for hearing loss.

  99. Pat

    I saw “La La”, heard “Ba Ba” – tried it by closing and opening my eyese while watching; very pronounced effect.

    Part of why it is hard to make sense of somebody over a phone: we depend more than we realize on the visual cues of language in addition to the auditory ones. Listening on a cell phone hands-free doesn’t stop this effect. And I think we listen more attentively to the phone rather than the radio because we are expected to respond. The radio is not any clearer: witness “‘Scuse me, while I kiss this guy!” it’s just that we aren’t expected to reply coherently.

  100. LS

    I just hear “ba ba” both ways. I wonder how much of that is from years of choral singing, where you have to train your ears to identify and match sounds?

  101. It doesn’t seem to work for me. I hear “ba ba” both ways, too. But then my wife is always saying I’m deaf.

  102. Jeffersonian

    Sounds exactly the same to me but the computer you’re using is more than 50% of how you will perceive it. My computer has handled youtube differently since they went to their new version and reduced screen options.

  103. Bomb, bomb, bomb, bomb Iran, bomb, bomb, bomb, bomb Iran….

  104. Pat

    Wow – Anchor, high-handed much?

  105. John Fleming

    I wonder what supposed category it puts me in if I’m not ‘fooled’ by the illusion, but can still see enough of the glitch to tell where one might be tripped up? Oddly enough, I think I might know why I’m not falling for it. As a long-time fan of video games, characters were once animated in two or three frames (mouth open/closed/somewhere in-between) as text scrolled. When games began using real audio voices, for a while, visuals didn’t catch up, meaning real voices often didn’t match the mouth movements. Perhaps growing up, I simply learned to disassociate and not rely too much on the visual cues?

    Others have noted that a career or hobby with video editing might impart a similar ability, and I think perhaps watching a lot of animated material might do the same.

  106. Joe Meils

    However, if you play the clip backwards, you distinctly hear:

    “John is dead… John is dead…”

  107. mitrax

    Ahahah, here’s something interesting:

    a few hours ago i first watched the video with headphones on, i couldn’t hear ANY difference whether my eyes were opened or closed, i heard “bah bah” in both cases, i had to read the comments to figure out what the illusion was supposed to be.

    i’ve just tried again, but on loudspeakers this time and… (don’t laugh) with earplugs (there’s a music festival today and a band is playing very loud just down my window, so i’ve been wearing them for 2 hours). Well, now i can hear the illusion: if i look at the video i clearly hear “dah dah”, if not, “bah bah”, funny thing is that was not intentional (watching it with earplugs).

    As it has been suggested by other commenters it’s probably directly related to the viewer’s audition. Ear plugs don’t attenuate sounds linearly they tend to act like a low pass filter, muffling the upper end of the spectrum but letting pass low frequencies, which is sort of what happens when aging (the max frequency you can hear becomes lower and lower with the years).

    For those of you who heard “bah bah” in both cases: if you have earplugs handy you should give it a try :)

    Phil should set up a poll to investigate this illusion further :p
    i bet the average age of those who ear “bah bah” in both cases (without impairing their hearing artificially that is) is lower than those for whom the illusion works.

  108. SpikeNut

    I was going to write in and say that Alan Alda explored this in Scientific American Frontiers a few years ago. But someone already beat me to it – and with links!

  109. rjpaus

    I can hear the difference, but so what? I think that our minds “sort out” what we hear so any ambiguity between, say “dah” and “bah” is smoothed out by context. It’s like reading – good readers don’t look at every word, they anticipate what is likely to come next through contextual clues. Why shouldn’t our hearing do the same? All we have in the video is a single, ambiguous syllable. But people very rarely say a single ambiguous syllable without other clues supplied by context, tone, pitch and expression.

  110. CR

    Tom Marking, that Sagan/Matrix vodeo was incredibly funny! Expectedly, the lip synch was off a little, but still fairly close. And even the hand motions matched!
    ************************
    For the record, for this thread’s video, I heard “Thah-thah, thah-thah, thah-thah” while looking, “Bah-bah, bah-bah, bah-bah” while not looking. If I closed my eyes & opened them during the video, the sounds would switch depending upon what I could see. (Still “thah” for sighted, “bah” for unsighted).

  111. CR

    “vodeo”?!

    Sigh… video.

  112. Anon

    Eyes open he says “Da da da”
    Eyes closed he says “I’ll kill you with this knife”

  113. jarno

    I’m in the da-da (looking) – ba ba (eyes closed) crowd.

    I wonder whether your either your language skills or your native language has anything to do with what you specifically hear? I am a native Finnish speaker, but I am quite as comfortable with English – I consider myself multilingual.

    In any case, an interesting phenomena. Thanks to the commenter waaay above for the wiki-link.

  114. Samantha Vimes

    Ba-ba, Ba, ba. That was what we heard, right. I could tell the lips were making completely different motions. It was like watching a low-budget dubbed movie, where the time and motion of lips to sound are completely unrelated.
    But seeing his lips move didn’t change what I heard at all.

  115. KerryK

    Perhaps not surprisingly, there do not appear to be many psycholinguists or phoneticians around here.

    Well, I’m one of the former breed. The McGurk effect has been well known and thoroughly described. It is an exquisite example of the brain’s use of different sensory modalities to extract information from the environment, and use that information to generate a mental representation of some event. It depends, among other things, on a feature of our perceptual systems known as categorical perception: The physical differences between the phonological instantiation of BA versus PA are in fact linear and continuous, taking place across about 20 milliseconds of “voicing” (the onset of vocal chord vibration). But we do not perceive the differences between BA and PA as continuous; there is a sharp cut-off in perception, with no middle ground. Interestingly, in the 70′s Patricia Kuhl at Univ. Washington showed that this perceptual mechanism which is clearly crucial for language is also present in chinchillas, suggesting that spoken human language is built out of phylogenetically old parts.

    McGurk described this sensory integration effect, showing how visual stimuli interact with our automated phonetic category assignments. We know now that this must depend on very fine-grained integration of activity in multiple brain regions that do not have direct links to one another, such as the hippocampus and the primary visual cortex. Individual differences in the wiring of such networks might help explain why some people don’t experience the effect in the demo, but we don’t know much about this yet.

  116. Samantha Vimes

    Oh, yeah, my computer’s a little old, so the mouth and sound didn’t synch up anyway. So I wasn’t distracted by the motions, even when watching.

    And I do watch enough badly dubbed movies that, yeah, I’m used to tuning it out. I wonder if fans of cheap Asian sci-fi have built up an immunity. Mothra help us!

  117. Peter

    Can’t you hear? He clearly says “Christ is coming…”

  118. Okay, first I heard lala, but watching again it became ahhdam with a short a. Interestingly, it persisted, unlike optical illusions.

  119. ME

    So the difference is between hearing DaDa and BaBa?

    What’s the big deal about that?!?!?! I was expecting the difference between “DaDa” and “Rotational Equilibrium”

    Sorry.. just not impressed. lol

  120. Chalk me up as another “I don’t see what’s interesting” viewer here…

    Off-topic, but:

    @Yojimbo: “What do you hear? It reminds me of the ambiguous sound the Japanese use for our R, L and D.”

    Huh? Japanese has a perfectly normal D sound, the same as English. It doesn’t have an exact equivalent of the English R or L; it does have a sound *not* found in (American) English that can make do as a way to express R or L. But that’s its own clear sound, and isn’t “ambiguous” in the least.

  121. I hear “ba ba” either way. I did notice that the video was in disagreement with the sound, but that was all. No change of sound at all.

    Perhaps an effect of watching too much dubbed-into-German movies and TV shows? I’m quite used to seeing lips that don’t agree with the sound.

  122. Ah, thank you Santiago for delivering a virtual haircut to my office. That was quite enjoyable, but I still can’t see the difference :-)

    BTW, I’m in the Ga Ga, Ba Ba camp, though I was confused for a bit trying to find some meaning in the ‘word’ being said. Then I started reading the comments and finally understood what I was supposed to be noticing. – g^2

    P.S. Glad you’re having a good time a TAM6 – I finally got around to twittering. . .

  123. purple velvet

    Hi folks..
    I’m a member of the 2% minority.. I’ve got my theory on this.

    - I’m not a native english speaker, so it sounds much more like ” papa”, said with a foreign accent to me ( a word, in french that means ” daddy”, so.. I can relate the sound to a “real” allday word, so no matter what, I hear this word.. The video seems to be “gaga” wrongly dubbed to me, but I still hear the sound “papa”

    - I’m a musician, so the sound comes first to me, and the video can’t trick me, as the visual perception isn’t as strong as audio. My ears dominate my eyes.

    But I must admit that , on a longer example, as you posted some weeks ago ( featuring the choir singing what could be a Purcell or Haendel’s Anthem with fake subtitles), I can easily be tricked: My english skills aren’t achieved enough to “fight” the subtitles.

    So greetings from France to everyone,from a geek-lady passionated by Languages, etymology, and anything that involves sound.

  124. jole

    I closed my eyes and heard, “No, you’re off YOUR case!”
    Must be the McGarnagle effect.

  125. Dave

    Interesting… but I was confused because phil said the guy was saying a single word… da da da da da da or ba ba ba ba ba ba…. I wasn’t considering that string of sounds a “word”…. now I see that he was just repeating a single sound…. makes more sense! :-P

  126. jole:

    I closed my eyes and heard, “No, you’re off YOUR case!”
    Must be the McGarnagle effect.

    “Perhaps I should turn one of you into a stopwatch.” – The McGonagall effect.

  127. drow

    weird. when i close my eyes, its “ba ba ba”, but when i look, it’s “da da da” and bill gates and steve ballmer are driving around in a small car pursued by a baby from outer space.

  128. Shoeshine Boy

    I heard “Ba Ba” both times, but my brain immediately recognized that the mouth movements were wrong.

  129. anonymous canuck

    Neat. I knew of this from lingusitics but never saw it before.

    Native English speaker “hearing” da .v. ba.

    Strangely, if I try it with one eye closed and then the other I “hear” it ever so slightly differently. But only on one eye. The other eye is the same as both eyes open. I wonder if that has anything to do with hand-dominance?

  130. Kingthorin

    Meh, I don’t get it. No matter how I look (or don’t look) or turn my head, it never sounds like anything even remotely English. It seems there’s lots of “Da-Da” going on….. hurray how nifty and exciting

  131. Yojimbo

    Traveler -

    “Huh? Japanese has a perfectly normal D sound, the same as English. It doesn’t have an exact equivalent of the English R or L; it does have a sound *not* found in (American) English that can make do as a way to express R or L. But that’s its own clear sound, and isn’t “ambiguous” in the least.”

    Not to belabor an off-topic point (and I see you are a fluent speaker), but, isn’t it true that in Japanese there is no “D” sound per se at all? Or any other pure consonant, for that matter – only consonant/vowel combinations (except the “n” sound). Their sound of “dah” is exactly like ours, but I don’t believe there is an equivalent of “dee” – is there a “doo”?

    Anyway, what I meant by ambiguous is that it would be very hard for an English speaker to hear a difference in the Japanese pronunciation of “ray”, “lay”, or “day” (or, in this case, “rah”, “la” and “dah”).

  132. Dick Dawkins

    Sorry, I couldn’t focus.

    I heard Charlie Brown’s teacher.

  133. Lo'ihi

    With eyes open, I hear in-between sound of dah, dah and bah, bah. Withe eyes closed, strangely I can barely hear ‘star dust’.

  134. In the comment posted on June 20 at 2:46pm, Sili wrote, “I’d like to see Language Log take this on.”

    Well, they did. Way back in 2004. Here are some of the posts in which they mentioned it:

    http://158.130.17.5/~myl/languagelog/archives/000637.html
    http://158.130.17.5/~myl/languagelog/archives/000639.html
    http://itre.cis.upenn.edu/~myl/languagelog/archives/001706.html

    (Some of the links in those posts are now broken, sorry.)

  135. GreginVancouver

    @Yojimbo & Traveller:
    Funnily enough I first saw this on a Japanese science program (Tameshite Gatten?) a few weeks ago and my Japanese wife and I saw “ma” and “da” but heard “ba” (The Japanese person on the screen was voicing “ma”)

    Santiago: Thank you for freaking me the hell out. I’ve heard 3D sound demos before but that was just too scary, especially when they put the bag over your head.

  136. I’m a radio broadcast engineer by trade and have always been audio oriented, so I heard the sound the same way, with and without watching the video. Blind friends have told me that I’m more audio oriented than most sighted people. So I think the difference is how audio oriented the person is.

  137. Randolph Garrett

    I have a severe hearing impairment and have taken lip reading classes which due to my visual difficulties I can’t do well.

    But what I do know, and this is very much like it, is that this is what is called “speech reading” where I use both my minimal hearing and focus on the face trying to lip read which then adds together and suddenly helps me to start getting words that I otherwise would miss.

    I’ll have to look up the McGurk effect to see if it’s speech reading or something else.

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

ADVERTISEMENT

See More

ADVERTISEMENT
Collapse bottom bar
+

Login to your Account

X
E-mail address:
Password:
Remember me
Forgot your password?
No problem. Click here to have it e-mailed to you.

Not Registered Yet?

Register now for FREE. Registration only takes a few minutes to complete. Register now »