Eagle-Eyed Autism? No.

By Neuroskeptic | June 29, 2011 9:00 am

An interesting and refreshing paper from Simon Baron-Cohen’s autism group from Cambridge. The results themselves are pretty boring – they found that people with autism have normal visual acuity.


But the story behind it is rather spicy.

Back in 2009, a Cambridge group – different authors, but led by “SBC”, published a report claiming that people with autism have exceptionally acute vision. Their average visual acuity was claimed to be 2.8

On this scale, 1.0 is defined as normal, and a sharp-eyed young adult with excellent eyesight would get about 1.5. 2.8 means nearly three times as good. Which is, literally, superhuman – a bird of prey would be happy with that. The paper was titled “Eagle-Eyed Visual Acuity In Autism”.

However, what followed was straight out of the Book of Obadiah“Though you soar like the eagle … from there I will bring you down, sayeth the Lord”. Or in this case, sayeth two experts in visual acuity research, Bach and Dakin, whose qualifications included the fact that they wrote the software used in the original study, which is online here.

They wrote a knock-down critique, arguing that the results were a result of using the wrong settings, which meant that the task was extremely easy. In fact, even perfect performance would only correspond to an acuity of less than 1.

You could never make a test so hard that it would require an acuity of 3.0 on a standard computer. Pixels are just too big. A single pixel is easy to spot, for someone of normal-ish vision. The only way to make it harder would be to use a special, extremely high-res monitor, or to get people to sit a long way from the screen.

So how did a result of nearly 3.0 come out? Because they also turned on data extrapolation, basically saying that if you really aced the easy task, you’d probably do quite well on a harder one. This might be sensible in some situations, but it breaks down when the task was so easy. The autistics seemed to have super vision because they got, say, 99% right, as opposed to 98%.

Yet the present paper represents a happy ending as it’s written by a combined team of Cambridge people, and Bach and Dakin as well, although the lead authors of the original weren’t on it. This time, they used appropriate methods – they got people to sit 4 meters from the screen. To be extra sure, they also gave everyone an eye exam before testing.

And they found no difference at all. The present paper is heartening – rather than grimly sticking to their guns, they admitted their error.


This story should however serve as a cautionary tale; I previously wrote about the fact that in science, a little mistake can cause a lot of problems. This is one of those cases, although arguably there were two seperate mistakes, but one, the extrapolation, was only a problem because of the main mistake, the big pixels.

ResearchBlogging.orgTavassoli T, Latham K, Bach M, Dakin SC, & Baron-Cohen S (2011). Psychophysical measures of visual acuity in autism spectrum conditions. Vision research PMID: 21704058

CATEGORIZED UNDER: autism, methods, papers, science
ADVERTISEMENT
  • Anonymous

    Let me get this straight: when they wrote the original paper, not once did the results alert them that maybe something wasn't quite right? They never consulted anyone with even a basic knowledge of vision testing? Not a single person who reviewed the paper pointed out that it was a load of crock?

    Even if they admitted their mistake, it's not really a heartening story, because the arrogance in the original paper is baffling.

    It's like if they let the subjects run 5 meters, noticed slightly better results in the autistics, and interpolated the results to determine that autistic people are The Flash.

  • http://www.blogger.com/profile/15118040887173718391 deevybee

    Original, wildly exciting but wrong, paper gets in Biological Psychiatry, impact factor 8.67
    New, boring correction gets in Vision Research, impact factor 2.33
    Needless to say, the original paper would never have made it past the referees in Vision Research.

  • http://www.blogger.com/profile/06647064768789308157 Neuroskeptic

    Certainly, Biological Psychiatry have a lot to answer for.

  • http://www.blogger.com/profile/11279402169161555639 Michelle Dawson

    See also non-replication published last year, reported by Armando Bertone's lab (down the hall).

  • Anonymous

    Biological Psychiatry have a lot to answer for for A LOT of papers. The journal is a joke.

  • http://www.blogger.com/profile/07811309183398223358 Zen

    Now this is how things are supposed to work in science. Go replication!

  • http://www.blogger.com/profile/09013133419305271189 Michelle Greene

    Great post. There's something a little strange about the graph, though – the median for both groups is around 2. Doesn't this mean everyone in their new study has twice the average acuity?

  • http://www.blogger.com/profile/05660407099521700995 petrossa

    I can shed some light on this misunderstanding i believe.

    Talking about HFA since i can't talk much to deep autists HFA is accompanied by a heightened sense of detail. It's not that our eyesight is better, we just see more details that others escape.

    So all these tests are targeting the wrong part.

    What one should do is let HFA's and NT's watch a action packed movie.

    During the movie they have to record what they see.

    Then you get results.

    If i watch a movie ( a bit hazy because my glasses aren't quite right) of a highspeed carchase full of fast cuts and action it strikes me that the plates on chased car front and back don't match.

    Things like that. Which makes it hard to watch a movie since every little detail is noticed. Plotholes, in-continuity, little scars of cosmetic surgery,a bird flying in the background, etc etc.

    In daily life its the same deal. There is no filter. Pretty annoying at times.

  • Anonymous

    If they made an error in the first study, shouldn't they be retracting study 1 rather than publishing study 2 and being rewarded with another line on the cv?

  • Anonymous

    Are you expecting the same people who wrote this line: “Individuals with ASC have significantly better visual acuity (20:7) compared with control subjects (20:13)-acuity so superior that it lies in the region reported for birds of prey.” to have the sense to call for a retraction?

  • Sue Gerrard

    Did they re-run the original participants? If not, then there's the risk that, methodological errors notwithstanding, the original experimental group could have eagle-eyed acuity but the current group don't.

    Also, even if the results in the second paper are more accurate, I don't think one can conclude that 'people with autism have normal vision' – I've come across several who most certainly don't and self-reports suggest that visual anomalies are not uncommon.

  • http://www.blogger.com/profile/06647064768789308157 Neuroskeptic

    Sue: Good point, I changed that sentence. This doesn't say anything about vision in general, just about visual acuity, i.e. the ability to focus and resolve very small objects.

    As petrossa says there is also the more general issue of noticing details and I think there's a lot of evidence for that.

  • Sue Gerrard

    So how does the noticing *details* work? What's the mechanism?

    I guess what I'm saying is that just because one group of people diagnosed with autism doesn't have visual hyperacuity it doesn't follow that no one diagnosed with autism has visual hyperacuity.

    Self-reports include descriptions of actively using the parafoveal area (peripheral vision) to avoid discomfort caused by foveal focus. How do we know that this discomfort isn't caused by visual hyperacusis?

  • Anonymous

    But the point is that the original test was so flawed that they might as well have measured the subjects' pinky toes for all it told them about visual acuity. Sure, the original group might have had superior acuity, but there's no way of knowing that, and since the correctly executed study showed no difference, the hypothesis doesn't have much going for it right now.

    Of course, this is the place for the old cliché about more research needed.

  • Sue Gerrard

    Indeed. Unfortunately any hypothesis involving “autism” doesn't have much going for it, because of the heterogenity involved.

  • http://www.blogger.com/profile/05660407099521700995 petrossa

    Sue

    I totally agree that anything involving studies about autism is a moving target.

    I am a moderator on a well visited HFA board and from the overwhelming anecdotal confluence of stories involving sensory processing to me it's very evident that at least true HFA have an extremely heightened sensory sensitivity.

    Any sense. Vision it is the most obvious, there is general consensus amongst the posters they all have this issue with seeing way too much detail.

    But touch, smell (therefore taste), hearing are also very acute.

    It's as often a nuisance as a good thing. Since you can't switch it off you tend to get nauseous a lot but on the other hand you can be very good at tasks involving the senses.

    The main reason for young HFA's (and maybe deep autistics) for throwing tantrums and the like is the overstimulation of the senses.

    There is not one poster who doesn't avoid crowds, noisy places because of the bombardment of sensory input.

  • http://www.blogger.com/profile/15225859145004971487 Jon Brock

    I've just been looking at the original paper and, to be fair, if I'd been asked to review it, I would probably have recommended accepting it. The explanation they give in the discussion is pretty vague and incoherent but the methodological flaws are not obvious to me as a generic autism researcher rather than a vision scientist.

    It's a nice illustration of the limitations of peer review and the fact that acceptance in a journal is not necessarily a sign of quality. As Dorothy pointed out, it's very unlikely that the original paper would have been accepted by Vision Research because it would have gone to at least one vision expert, who would have quickly pointed out the problems.

    In general, I now tend to favour sending my autism papers to journals that are relevant to the discipline of the actual study rather than specialist autism / child psychology journals. I started doing this because I was sick of our papers getting rejected based entirely on the fact that we hadn't matched on some spurious variable or hadn't rediagnosed all our participants with the ADI and ADOS. But the real positive is that you almost always get some really knowledgeable reviewers who can point out potential issues that you'd never have thought of unless you'd been in the field for years.

  • http://www.blogger.com/profile/05660407099521700995 petrossa

    ADI and ADOS are circular confirmation systems of the DSM criteria and as such pretty useless.

    If you setup a bunch of diagnostic criteria based on behavioral observation (pretty unscientific) then over the decades create further criteria/'refinements' and diagnostic systems based on those original iffy criteria you'll end up with a total mess.

    DSM should be scrapped. Then a thorough delineation between physical and non-psychical mental anomalies should be made using any scan system (except fMRI ofcourse).

    Once you have a baseline of true neuro-developmental issues and people with a bad childhood you can start to try and classify them.

    My guess for the subject at hand: Schizophrenia, autism, aspergers, CC agenesis etc belong to the same category.

  • Sue Gerrard

    Petrossa has hit the nail on the head.

    My only reservation is that in children it might not be possible to detect any differences via brain scans if the developmental anomalies were caused by issues involving sense organs or neurochemical factors, because they might not yet have resulted in observable differences in brain structure or function.

  • pseudonymoniae

    Petrossa's plan seems to assume that its actually possible to effectively classify mental illnesses based upon such criteria. If we can't identify a person as having a given illness using a scan, does his/her illness no longer exist? I think you're ignoring the fact that “illnesses” are defined by whether or not they affect life outcomes. Under this scheme, somebody whose scan indicates they have an illness but who doesn't require any kind of treatment might be classified as “schizophrenic” or whatever arbitrary name we might cook up, even without expressing any of the corresponding symptoms. And I'm not even all that sure what “non-physical mental anomalies” are supposed to be, much less how to identify them.

    I don't object to distinguishing between illnesses using some diagnostically useful scanning technique, but I do think that scrapping the DSM is a bad idea. Perhaps, de-emphasizing the importance of the DSM might be better?

  • Sue Gerrard

    pseudonymoniae: what function does the DSM actually fulfill in the case of disorders with unknown aetiologies?

  • pseudonymoniae

    @Sue

    Well, that's just the point. If the aetiology of a disease is unknown, then we need some means of identifying people who suffer from the disease. Now, I'm not saying that the DSM offers an ideal system of classification, nor even one that is necessarily valid. But I am skeptical of the claim that we have sufficient methods to identify and classify mental illness without using behavioural measures.

    I'm not saying that we shouldn't attempt to develop more valid methods of distinguishing between mental illnesses. But it is clearly wrong to claim that we currently have the appropriate scanning methods to completely replace the DSM's function.

  • Sue Gerrard

    I completely agree about scanning; I'm just concerned about how a classification system of doubtful validity is used.

    Or why it's still being used.

  • http://www.blogger.com/profile/05660407099521700995 petrossa

    I agree with both the lack of proper tools to diagnose a neuro-developmental issues in a gestating foetus. That level of understanding how the connections propagate, nor therefore a mechanism for observing it has not yet been achieved.

    I do think it's possible, once neuro-scientists latch on, to examine the white matter during infancy. Here the behavioral component is a must. The child behaves odd, the oddness conforms to a certain pattern, and one then studies the brain for clues.

    Over time a general developmental structure can be educed and after that a disorder can be defined.

    The problem the profession is faced with is the decades of lack of progress caused by the stubborn adherence to a wholly subjective collection of ancient criteria.

    Talent in general is few and far between. One needs a very powerful mind indeed to overcome the corset of current educational systems.

    Most people studying psychiatry, whilst having their own merits, will not be that talented they can avoid being indoctrinated with the idea's, theories and assumed facts installed on the basis of the DSM/DSM equivalent.

    It's a circular process. Students inhale DSM and exhale DSM+ and thus become later the editors of DSM XV.

  • http://www.blogger.com/profile/05660407099521700995 petrossa

    we're getting there
    http://www.ncbi.nlm.nih.gov/pubmed?term=21718979

    Now for a more refined method

  • http://www.blogger.com/profile/00510187022062895431 Overdo it, I encourage it!

    Yeah, like are you saying that the normal vision in autistic people was misdetected and it's not normal? And are you saying that larger objects look larger to them but smaller objects though they look larger than normal are quite likely to require a microscope like everyone else even if it looks bigger? Fascinating! Autism isn't autism any more, it's called Oppositional Defiance Disorder.

  • http://www.blogger.com/profile/06647064768789308157 Neuroskeptic

    Or Oppositional Eagle-Eye-Ance Disorder 😉

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Neuroskeptic

No brain. No gain.

About Neuroskeptic

Neuroskeptic is a British neuroscientist who takes a skeptical look at his own field, and beyond. His blog offers a look at the latest developments in neuroscience, psychiatry and psychology through a critical lens.

ADVERTISEMENT

See More

@Neuro_Skeptic on Twitter

ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Collapse bottom bar
+