Google’s Artificial Intelligence Masters Classic Atari Video Games

By Toby Walsh, NICTA | February 26, 2015 5:04 pm

atari-brain

Think you’re good at classic arcade games such as Space Invaders, Breakout and Pong? Think again.

In a groundbreaking paper published yesterday in Nature, a team of researchers led by DeepMind co-founder Demis Hassabis reported developing a deep neural network that was able to learn to play such games at an expert level.

What makes this achievement all the more impressive is that the program was not given any background knowledge about the games. It just had access to the score and the pixels on the screen.

It didn’t know about bats, balls, lasers or any of the other things we humans need to know about in order to play the games.

But by playing lots and lots of games many times over, the computer learned first how to play, and then how to play well.

A Machine That Learns From Scratch

This is the latest in a series of breakthroughs in deep learning, one of the hottest topics today in artificial intelligence (AI).

Actually, DeepMind isn’t the first such success at playing games. Twenty years ago a computer program known as TD-Gammon learned to play backgammon at a super-human level also using a neural network.

But TD-Gammon never did so well at similar games such as chess, Go or checkers.

In a few years time, though, you’re likely to see such deep learning in your Google search results. Early last year, inspired by results like these, Google bought DeepMind for a reported $400 million.

Many other technology companies are spending big in this space. Baidu, the “Chinese Google”, set up the Institute of Deep Learning and hired experts such as Stanford University professor Andrew Ng. Facebook has set up its Artificial Intelligence Research Lab which is led by another deep learning expert, Yann LeCun. And more recently Twitter acquired Madbits, another deep learning startup.

The Secret Sauce of Deep Learning

Geoffrey Hinton is one of the pioneers in this area, and is another recent Google hire. In an inspiring keynote talk at last month’s annual meeting of the Association for the Advancement of Artificial Intelligence, he outlined three main reasons for these recent breakthroughs.

First, lots of Central Processing Units (CPUs). These are not the sort of neural networks you can train at home. It takes thousands of CPUs to train the many layers of these networks. This requires some serious computing power.

In fact, a lot of progress is being made using the raw horse power of Graphics Processing Units (GPUs), the super fast chips that power graphics engines in the very same arcade games.

Second, lots of data. The deep neural network plays the arcade game millions of times.

Third, a couple of nifty tricks for speeding up the learning such as training a collection of networks rather than a single one. Think the wisdom of crowds.

What Will Deep Learning Be Good For?

Despite all the excitement about deep learning technologies, there are some limitations to what it can do.

Deep learning appears to be good for low-level tasks that we do without much thinking. Recognizing a cat in a picture, understanding some speech on the phone or playing an arcade game like an expert.

These are all tasks we have “compiled” down into our own marvelous neural networks.

Cutting through the hype, it’s much less clear if deep learning will be so good at high level reasoning. This includes proving difficult mathematical theorems, optimizing a complex supply chain or scheduling all the planes in an airline.

Where Next for Deep Learning?

Deep learning is sure to turn up in a browser or smartphone near you before too long. We will see products such as a super smart Siri that simplifies your life by predicting your next desire.

But I suspect there will eventually be a deep learning backlash in a few years time when we run into the limitations of this technology. Especially if more deep learning startups sell for hundreds of millions of dollars. It will be hard to meet the expectations that all these dollars entail.

Nevertheless, deep learning looks set to be another piece of the AI jigsaw. Putting these and other pieces together will see much of what we humans do replicated by computers.

If you want to hear more about the future of AI, I invite you to the Next Big Thing Summit in Melbourne on April 21, 2015. This is part of the two-day CONNECT conference taking place in the Victorian capital.

Along with AI experts such as Sebastian Thrun and Rodney Brooks, I will be trying to predict where all of this is taking us.

And if you’re feeling nostalgic and want to try your hand out at one of these games, go to Google Images and search for “atari breakout” (or follow this link). You’ll get a browser version of the Atari classic to play.

And once you’re an expert at Breakout, you might want to head to Atari’s arcade website.

 

Here, DeepMind plays a Pong-style game. After 600 episodes the computer finds and exploits the optimal strategy in this game, which is to make a tunnel around the side, and then allow the ball to hit blocks by bouncing behind the wall.

The Conversation

This article was originally published on The Conversation.

Top image courtesy Google DeepMind 

CATEGORIZED UNDER: Technology, Top Posts
MORE ABOUT: computers
ADVERTISEMENT
  • YeahRight

    Yeah… that’s not artificial intelligence, either.

    • JasonKaler

      So how would you categorize this?

      • YeahRight

        A learning algorithm. People have been studying those to death over the years and they are very good at solving certain types of control problems like these games. Try teaching chess to one of these and the results will be rather disappointing. Try teaching mathematics or human languages and there will be no results.

        • PlayModeOn

          It’s a start….

          • YeahRight

            To a computer that can beat you at a trivial game like Asteroids? Certainly. A few days ago I was beaten 10 times out of 10 by a 1kbyte chess program. Does that make that chess program intelligent? No, it makes me suck at chess.

          • PlayModeOn

            You do not need write up a paragraph explaining it to me.

          • YeahRight

            I didn’t have to or I don’t have to? :-)

        • JasonKaler

          “learning algorithm”s fall under the category of AI.
          So do steering algorithms.
          Just because it disappoints you because it can’t learn a language, does not mean it does not fall under the category of AI.
          Have YOU ever written anything this intelligent?
          Mice can’t learn to play chess and yet we call them intelligent.

          • YeahRight

            OK, I have no idea what college/university teaches that nonsense, but if yours did, it’s time to ask for your money back.

            Have I ever written a learning algorithm that can do something like this? Yes, I wrote a genetic algorithm once that could learn to become a complete word processor without there being any knowledge about what a word processor is being in its representation in the beginning. Took me about a week. Seriously man, you need to learn to distinguish trivial stuff from the hard jit.

          • Kyli Rouge

            We don’t know what’s going on under the hood. This might well be an ANN or GA.

          • YeahRight

            Both ANN and GA are trivial learning algorithms, neither is AI.

          • Kyli Rouge

            Actually, they are AI.
            A neuron is a trivial chemical reaction, and is not intelligence.
            ANN has the ability to be very intelligent, given enough neurons, layers, and speed.

    • Pit Kid Icarus

      It’s like amiibo? Except smarter. That’s it, Skynet confirmed!!

  • YeahRight

    No, it just shows a guy who is full of himself. That’s neither I nor AI.

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

The Crux

A collection of bright and big ideas about timely and important science from a community of experts.
ADVERTISEMENT

See More

ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Collapse bottom bar
+