We Folded: AI Bests the Top Human Poker Pros

By Nathaniel Scharping | January 31, 2017 1:46 pm
864

The Brains vs Artificial Intelligence competition at the Rivers Casino in Pittsburgh. (Credit: Carnegie Mellon University)

Roughly a year ago, to the day, Google researchers announced their artificial intelligence, AlphaGo, had mastered the ancient game of Go. At the time, Discover wrote that there was still one game that gave computers fits: poker.

Not anymore.

Late Monday night, a computer program designed by two Carnegie Mellon University researchers beat four of the world’s top no-limit Texas Hold’em poker players. The algorithm, named Libratus by its creators, collected more than $1.5 million in chips after a marathon 20-day tournament in Pittsburgh. The victory comes only two years after the same researchers’ algorithm failed to beat human players.

Tough Nut to Crack

In the past few decades, computer scientists’ algorithms have surpassed human prowess in checkers, chess, Scrabble, Jeopardy! and Go—our biological dominance in recreational pastimes is dwindling. But board games are played with a finite set of moves, the rules are clear-cut and your opponent’s strategy unfolds on the board. Computers are well-adapted to sort through and make the most optimal choices in these logical, rules-based games.

Poker was seen as a stronghold for human minds because it relies heavily on “imperfect intelligence”—we don’t know which cards our opponents hold, and the number of possible moves is so large it defies calculation. In addition, divining an opponent’s next move relies heavily on psychology. The best players have refined bluffing into an art form, but computers don’t fare very well when asked to intuit how humans will react.

These hurdles were obviously no match for the improved algorithm designed by Tuomas Sandholm and Noam Brown. While they haven’t yet released the specific details of their program, it seems that they relied on the well-worn tactic of “training.” Libratus ran trillions of simulated poker games, building its skills through trial and error, until it discovered an optimal, winning strategy. This allowed the AI to learn the nuances of bluffing and calling all by itself, and meant that it could learn from its mistakes.

“The best AI’s ability to do strategic reasoning with imperfect information has now surpassed that of the best humans,” Sandholm said in a statement.

Better Every Day

Sandholm says that the Libratus would review each day’s play every night and address the three most problematic holes in its strategy. When play began the next day, the human players were forced to try new strategies in their attempt to trick the machine. The poker pros would meet every night as well to discuss strategies, but their efforts couldn’t match the processing power of the Pittsburgh Supercomputing Center’s Bridges computer, which drew upon on the equivalent of 3,300 laptops worth of computing power.

Libratus seemed to favor large, risky bets, which initially made the human players balk. They soon learned that it was best to try and defeat the AI early on in a hand, as that’s when the most cards are unseen and uncertainty is greatest. As more cards are flipped and decisions made, the computer was able to further refine its decision making.

The algorithm isn’t limited to poker either. While this version of the program was trained specifically on the rules of Texas Hold ‘Em, it was written broadly enough that it could conceivably learn to master any situation that contains imperfect information, such as negotiations, military strategy and medical planning.

Libratus isn’t quite ready for the World Poker Tour yet. The version of the game it played only included two opponents at a time, unlike most tournaments. Games with more players compound the number of variables at play, making it significantly more difficult for a computer to choose the best course of action.

So when it comes to joining the poker table with Libratus, heed the immortal words of Kenny Rogers: Know when to walk away. Know when to run.

CATEGORIZED UNDER: Technology
ADVERTISEMENT
  • http://www.mazepath.com/uncleal/qz4.htm Uncle Al

    What defeats a learning artificial intelligence? Entropy. Play irrationally to mistrain it with many small bets, then go in for the kill.

    • darryl

      I’m guessing the players were doing this to some extent to throw it off. At the same time, one must not make any wrong moves or the computer will exploit the hell out of you. It seems as though eventually the computer managed to stay a step ahead of the humans.
      Kind of reminds me of that computer program that will always beat a human at guessing heads or tails. The best one can do over an infinite number of coin flips is to get a tie, and that’s only by using perfectly random numbers which humans are terrible at.

      -d

      • lump1

        A computer will not beat a human at guessing heads or tails. Each throw is independent of the rest. There is no benefit to randomizing your guesses, because previous throws don’t affect the next one. If you guessed heads each time, you’d still get your 50%.

        • darryl

          It sure will because humans are terrible at randomness. The computer can exploit this.
          Now if a human were to actually toss a coin and have a computer guess, that would be different and it should be 50/50 if the coin is fair. If the human just makes it up, the computer is going to win because it’s not independent despite what the human thinks.

          -d

        • Inimical Jim

          I don’t have the source in front of me but I remember reading about a professor splitting a class in half and having one half flip a coin 100 times and recording the results and the other half making up results. He could tell the phonies because there were more HTHTHT trends when the actual flips looked more like THHHTTH.

    • OWilson

      Do what Vegas does, ply it with alcohol :)

      • Robert Kolker

        give the suckers a little free booze, a little free food and let them win ever now and again but seldom and with small pay-offs. That will keep the suckers at the tables and bring them back again and again.

        • OWilson

          The odds are slightly in Vegas favor, but their main gravy comes from the egos, sometimes alcohol, or female fanned, induced to “let it ride”, or “I feel lucky, watch me draw to 17” :)

          • Robert Kolker

            Words of doom, dismay and destruction —- I feel lucky. I am on a roll. The goddess of fortune smiles on me etc. etc.

  • Robert Kolker

    I object. Intelligence is general. Intelligent beings can solve all kinds of -different- problems. Building a machine to play poker (no doubt by embedding some kind of Bayesian inference scheme) is very clever programming. The only intelligence being exhibited here is the intelligence of the algorithm’s designers and programmers. Also a tip of the hit to the engineers who designed and made the chips for the machine to carry out the algorithm.

    • gonzalez.aaron

      I profited 104000 dollars last year by doing an on-line job a­­n­­d I did it by wo­rking in my own time f­­o­­r several hrs daily. I followed a money making opportunity I stumbled upon online and I am so thrilled that I was able to earn so much money on the side. It’s so newbie-friendly a­n­d I’m just so thankful that i discovered it. Check out what I do… http://www.wzurl­.­me/tPSR1w

    • http://secure93.com Anton Andrews

      I sᴛarted ᴡorkͺnɡ ᴏnlɩпe‚ by ᑯ೦ӏϖƍ vari໐ᴜs bаsıc јobs ᴛһat on|y reԛuiᴦes ſrom you ԁеƽkтop or lapt໐ρ cංᴍputer anᑯ interneт αcᴄeѕs αռd I c๐uIdϖ’ᴛ b℮ haррier..܂ Ѕι᙮ moոᴛհs һɑve passed since i ѕᴛaᴦτеd τhᎥs anᑯ ı ɡοт ρаӏᏧ ƽo ſar тotaI oſ $Ʒ6‚О0ଠ۰.۰ ВasiϲІy ɩ ρrоfit close to $ȣ0/һoцr ɑnd vvогк fסг Ʒ‒Ꮞ hrs ංп daily Ƅaѕіs܂B℮ƽt pαrτ tσ ᴡһoǀ℮ тhᎥƽ ᴛհɪոƍ is ᴛhαᴛ you can deᴛermıne ʏ٥цr oառ ᴡoᴦκɪnƍ һ०ʋᴦs αnᏧ ᶌou ցet a ρaychеck aт τhe eϖd оſ eᴠery w℮℮k۔>›˃> RU.VU/6OmJE

  • https://www.bestonlinesportsbooks.info/ Andrew Scofield

    Uncle Al. You are correct will be very difficult to defeats a learning artificial intelligence in the near future.

  • B. Dickey

    Wait, this whole thing doesn’t read like the headline. For one, no, you can’t play poker with a computer, that’s not what the game is about. Second, the human players had to be limited because of the constraints of the machine. This isn’t intelligence, its still programming. Now, if that computer jumped up after winning in jubilation and grabbed and kissed the nearest waitress, I’d think differently, but this is nothing.

    • Tony Reno

      Exactly.

      That’s exactly the way I thought when I saw Watson on Jeopardy. If the machine has to be fed the data in binary form, and can’t even understand what the host is saying, it’s not playing Jeopardy.

      And if the machine isn’t picking up it’s own cards, and looking into the eyes of the opponents, and isn’t worried about paying it’s mortgage, or hoping to fund a new boat, it’s not playing poker.

  • Alan Lambert

    Just like I didn’t buy Chess couldn’t be beaten by an AI until Deep Blue beat Kasparov and Jeopardy! could be beaten until Watson came along, I will not buy an AI beating Texas Hold ‘Em until one wins the Main Event at the WSOP.

    Period.

    {Doveryai, no proveryai}

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

D-brief

Briefing you on the must-know news and trending topics in science and technology today.
ADVERTISEMENT

See More

ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Collapse bottom bar
+