“Brain Training” Doesn’t Work?

By Neuroskeptic | July 13, 2017 2:43 pm

Lumosity “brain training” games have no beneficial effects on cognition, according to a paper just published in the Journal of Neuroscience.

According to the authors, led by UPenn psychologist Joseph W. Kable, Lumosity “appears to have no benefits in healthy young adults above those of standard video games.”

In the study, 128 young adults were randomly assigned to either 10 weeks of Lumosity training, or a control condition: 10 weeks of playing normal, non-brain-based online videogames. The Lumosity group got much better at the Lumosity tasks over time, as shown by the Lumosity Performance Index:

kable_lumosity_2

However, neither Lumosity nor the control had any effect on the key outcome: executive function. Executive function refers to cognitive processes that regulate behaviour and help make reasoned (as opposed to impulsive) decisions. Kable et al. measured executive function with two decision-making tasks called delay discounting and risk sensitivity. Neither group improved on either task:

kable_lumosIn other words, mastering the Lumosity tasks didn’t generalize to better executive function, even though many of the tasks were designed to do just that.

On other cognitive measures, it was a similar story. The Lumosity group showed no better performance than the videogame group on tests of working memory, sustained attention, and other cognitive abilities.

Is it possible that Lumosity was “training the brain” but that somehow this neural effect didn’t manifest as better performance? It doesn’t seem likely. Kable et al. used fMRI to measure brain activation in the participants while they were performing the executive function tasks. This revealed no difference in brain activity between the Lumosity and control groups either.

In my view this is a very well-designed study. The use of an active control (videogames) will have helped to reduce placebo and other non-specific influences such as the Hawthorne effect. Importantly, the control group were instructed to keep to the same training schedule (5 sessions of 30 minutes each, per week) as the Lumosity group. Many previous studies of “brain training” used less engaging control treatments, or none at all.

Kable et al.’s study does have one main weakness though, which is that it’s just not very big. With 64 participants per group, this study wasn’t tiny, but in 2017 it’s pretty small. Seven years ago there was a similar study with n=11,430 (although it didn’t examine Lumosity tasks.) Lumosity might therefore argue that Kable et al.’s study lacked the statistical power to detect the benefits of their product.

This study would probably have been much bigger if the fMRI component hadn’t been included. MRIs are expensive. I’m not sure the MRI adds that much to the study, in all honesty. Then again, the authors might have struggled to get a grant without promising to do some fMRI: the allure of neuroimaging can have a powerful effect on funders.

It’s also notable that Kable et al. studied young and healthy adults. It’s possible that people with existing cognitive impairment might benefit more from “brain training”.

CATEGORIZED UNDER: fMRI, papers, select, Top Posts
ADVERTISEMENT
  • Pingback: 1 – “Brain Training” Doesn't Work?()

  • OWilson

    Practice does not always make perfect but, I believe that “training” the mind or body does produce beneficial effects.

    The athlete is an obvious example, but I remember years ago , when IQ Tests were a topic of the day, I bought the books, and did many online online tests to sere if I could raise by IQ score.

    Modesty prevents me from revealing it here, but I DID significantly improve my score.

    Perhaps my intrinsic intelligence was raised (some at Discover would argue against that rather vehemently :) or whether I just got better at test taking in general, perhaps understanding how the questions were formulated by the test setters, learning to see relationships between apparently disparate sequences and designs, and being more aware of the time limit impositions.

    I always figured you could easily learn to beat a Lie Detector in the same way.

    And I though it may affect physical and mental stress tests, for some folks who are stressed out by tests, anyway!

    • http://kapitano.blogspot.com/ Kapitano

      So you played a game, and got more skilled at it over time. If that’s all there is to general intelligence, it’s no more impressive than learning to touch type.

      • OWilson

        My point, precisely!

        • Erik Bosma

          Totally agree. It’s all about experience and memory.

    • George

      The best way to defeat a lie detector is to not feel guilt about lying or to really believe in your lies so honestly that they are not lies.

      As someone who has sat through a few polygraphs, I can say that guilt is something very innate and completely irrational. At least for me, attempts at frontal cortex control did not impact the responses my brain stem and thyroid were producing. My irrational guilt was always in control, no matter how much I thought about the questions asked. That mini electric piano I accidentally stole when I was 5; cheating one night when my grandmother asked me to spell words I had to learn when I was 6 or 7; the $6 I borrowed from my brother when I was 19 and never paid back, the MP3s I hadn’t bothered to remove from my computer in college; and the beer I bought that underage classmates in college drank got all mixed up with questions regarding my overall trustworthiness. Eventually I “passed” my lie detector tests, but I’m not sure people can train for them. My brain connections and chemistry likely were set when I was a child who learned “lying” is bad. I’m of the opinion that some people never learn that lying bad, and they pass lie detector tests with flying colors.

      As for online test, I don’t believe they improve cognitive function in people who regularly use their brains.

      • OWilson

        A couple things.

        My whole point is that that an IQ score is a just a (culturally biased?) test score and is NOT directly proportional to intrinsic intelligence in a human. I think we all here agree on that point.

        As for passing lie detector tests with all your apparent Freudian conflicts and suppressed guilt, I would just say that to pass a lie detector, the requisite skill consists mainly of being a psychopath, a sociopath, without conscience or guilt.

        You were brought up too well, in spite of your minor transgressions. :)

        • George

          Agreed on all points. Based on IQ scores, I’m a genius. In reality, I am not. I do very well on tests and my spacial intelligence is likely off of the charts. I can draw accurate maps from memory and observation. I can tell you when and where a video was filmed based on sun shadow angles and some other clues without using a protractor on the computer screen. Movie directors don’t seem to get it. I’ve gained functional fluency in languages by contact alone. I’ve also had trouble telling left from right quite recently, asked if an elevator was going down from the observation deck of a building, and been cheated from money in my pocket by “lowlifes.” Something in my brain works for IQ tests and map-making, just don’t ask for directions. : )

          • OWilson

            We apparently have something in common, especially maps, and filming locations.

            Europe during and just after the war was a dark place with a scarcity of local maps so at an early age I was drawing neighborhood maps just from memory and observation. I would even take different bus rides to fill in gaps! :)

            Our gloomy black and white existence was broken only by Hollywood movies at the local cinemas, and I was an avid fan. I became fascinated by the same shooting locations that appeared to pop up all the time, and tried to put them together.

            Fast forward to the 70s after I became nearly famous :), As I mentioned in another post, I was in California, and after a cursory pass at the studios, I was at Lone Pine, the Simi Hills, the Valley, even the infamous Spahn Ranch with my camera. Standing on the very rock beside the trail, that the “injun varmint” jumped from onto Hopalong Cassidy. San Fransisco with its Film Noir settings.

            All the familiar locations I had seen as a child in a darkened theater, and which sparked my desire to leave my meager surroundings.

            I should mention I am less than impressed by the Hollywood of today, which appears to me as just a bunch of hacks, turning out non-interactive video games.

            Of course, I’m getting old! :)

          • George

            : )

            You might be getting old, but wisdom always remains the same age, which was born in the year “Perfect.”

          • OWilson

            The “Year Perfect” for me was when they brought out Google Earth! :)

          • http://www.mazepath.com/uncleal/qz4.htm Uncle Al

            As for accumulating deficiencies of the elderly, we force feed them longevity while plundering their wallets to deny them happiness. We warehouse the pre-deceased at ruinous cost to the young, direct confiscation, squandered inheritance, and corruption.

          • Erik Bosma

            Yep. According to the texperts I, also, am a genius. !50. I wish I could trade that number in for something tangible. Finished my Grade 12 at age 34 after working in lumber, steel and oil in my teens and twenties. Worked several different jobs after I ‘graduated’ and I’m now 63, unemployed and with 3 years of university. Still don’t know what I want to be when I grow up. Maybe an anthropologist but probably I’ll be a corpse. Problem was, I have always been the square peg in the round hole. Could never relate to our society or any other society for that matter. Maybe northern India around the time of the Buddha or Baghdad during its heyday.

          • Erik Bosma

            Haha… talk about a Freudian slip. That should be 150 not !50, although I like the second one better.

        • jrkrideau

          I’m pretty sure one can learn to beat the polygraph operator. The polygraph seems to have no real validity as far as one can see. However a good polygraph operator probably can read a person very well.

          Witchdoctors in Africa are similar, I believe.

          Come to think of it I don’t know if anyone has even checked the polygraph’s reliability.

          The chapter in the US National Research Council’s report Strengthening forensic science in the United States: a path forward is interesting reading
          https://www.ncjrs.gov/pdffiles1/nij/grants/228091.pdf

          Actually just about every chapter has a tone of total incredulity. It gives the impression that “forensic science” is a oxymoron, to say the least.

          And yes, a natural polygraph beater would likely be a psychopath.

          • OWilson

            You raise an interesting point.

            The “Phlogiston Mind Reading Machine” that strikes fear in the guilty.

            A prop, no more effective than a crystal ball, which lends credibility to nothing more than a good personality profiler.

            Never thought of it that way before, Thanks!

          • jrkrideau

            Yes, ‘The “Phlogiston Mind Reading Machine” that strikes fear in the guilty’ is an excellent way to describe it.

            I once had a reply to my acerbic comment on a newsgroup about the polygraph saying how good polygraph operators were when playing poker. Duh.

  • jonathanpulliam

    I used to lap PS2 Gran Turismo 4’s Nurburgring in 6:30 w/ the Chaparral 2J. Just for heck of it I drove while playing Mozart’s Serenade from the Amadeus soundtrack CD and I feel that little oomph got my time down to 6:20.

    • stupid human

      I got a similar effect with Alive by Say Anything.

  • http://www.pbase.com/davidjl David Littleboy

    Two comments: Isn’t this the expected result? There have been very few cases where practice in one area (e.g. learning Latin, violin) produced even small improvements in any other area not closely related. (In that sense, that running, and also probably even brisk walking, improves overall cognitive performance is seriously kewl.)
    Also, that weakness isn’t that much of a weakness. If the effect of the brain training game is too small to see in an n = 64 trial, we’re done. A tiny
    effect that can only be seen in an enormous trial means the game isn’t worth spending the time or money on.

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      It’s not that surprising to you and me, but it might be surprising to people who pay for Lumosity!

      About the power, it all depends what size of effect you’d accept as a useful effect of Lumosity.

      Kable et al. had an 80% chance to detect an effect if it had a size of Cohen’s d=0.44. d=0.44 is a pretty substantial effect, and they had a 20% chance of missing it.

      For a d=0.3 effect, which is “small” (but still, it’s roughly equivalent to 5 IQ points which most people would be glad to have) they’d have a 60% chance of missing it.

      • William Holz

        That’s not a good time/benefit result, is it?

        Imagine what you could do if you spent that time doing something wacky like…oh, I don’t know…learning something new?

        :)

  • Christien Janson

    As usual the test persons are young and healthy. If only they would start testing with the people that could benefit, and not with the ones that make the tests the easiest. The world is full of solutions for people that need them, that have been thrown away because they did not work for healthy testpeople. So discouraging!

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      True, but Lumosity don’t advertise their products as “for cognitively impaired people only” as far as I know.

    • OWilson

      Why do they never show fat people on Bowflex or Elliptical machines? :)

  • JWrenn

    Is there a list of what games they played?

  • John Plass

    This study is begging for a Bayesian analysis. That would help us understand how much your sample size criticism matters. In fact, I’m surprised it was published without it.

  • RogerSweeny

    Most of us who comment here find it easy to believe these results and to look down on Lumosity. However, the study also calls into question common advice from much more respectable people. Namely, the advice by health professionals that to avoid (or at least slow down) dementia, you should engage in mentally challenging tasks. After all, there is a correlation between engaging in mentally challenging tasks and not being demented.

    But the direction of causation may well be from the latter to the former.

    • William Holz

      They didn’t study people who had or were on the fringe of having dementia here.

      I’m also not sure that Luminosity counts as mentally challenging, at least not even vaguely comparable to how challenging incorporating new ideas is.

  • Денис Бурчаков

    Lumosity games are so boring and repetitive, that I’m not sure, whether they qualify as games. Because, you know, games should be fun. I think, that Dwarf Fortress is a far better method of improving spatial orientation, memory and decision making. And Dwarf Fortress is full of FUN.

  • Macià Buades-Rotger

    Cool stuff! Just one comment: I think fMRI is not only alluring for funders, but for authors themselves. Neuroimaging journals are usually better ranked (citation- and impact factor-wise) than regular psychology journals, and the required sample sizes to publish in the former are way lower. Without neuroimaging and a bigger sample, this study might have been, say, a J Exp Psychol: General (IF 4.4), but add fMRI and you get a J Neurosci (IF 5.9) with less subjects. Plus, you show that you can also analyze neuroimaging data, which is good for your resume. I’m not blaming the authors here (the study is awesome), just pointing out yet another aspect of the perverse incentive structure in psychological science.

  • Sys Best

    Fun games do work on the brain, you have fun, get happy, you get a rush from competing, some satisfaction from winning, less if losing, learn a few new things. It may not make you smarter but you may benefit from a relaxation response similar to other activities.

  • Pingback: Psychology News – 14 July, 2017 – numerons()

  • Pingback: Psychology News – 14 July, 2017 – numerons()

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Neuroskeptic

No brain. No gain.

About Neuroskeptic

Neuroskeptic is a British neuroscientist who takes a skeptical look at his own field, and beyond. His blog offers a look at the latest developments in neuroscience, psychiatry and psychology through a critical lens.

ADVERTISEMENT

See More

@Neuro_Skeptic on Twitter

ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Collapse bottom bar
+