How Probability Works

By Sean Carroll | September 5, 2011 9:25 am

From Barry Greenstein’s insightful poker book, Ace on the River:

Someone shows you a coin with a head and a tail on it. You watch him flip it ten times and all ten times it comes up heads. What is the probability that it will come up heads on the eleventh flip?

A novice gambler would tell you, “Tails is more likely than heads, since things have to even out and tails is due to come up.”

A math student would tell you, “We can’t predict the future from the past. The odds are still even.”

A professional gambler would say, “There must be something wrong with the coin or the way it is being flipped. I wouldn’t bet with the guy flipping it, but I’d bet someone else that heads will come up again.”

Yes I know the math student would really say “individual trials are uncorrelated,” not “we can’t predict the future from the past.” The lesson still holds.

Happy Labor Day, everyone.

CATEGORIZED UNDER: Entertainment, Mathematics
  • Spencer

    On Thursday I composed a musical soundscape called “Cosmic Variance.” Now what are the odds?

  • Fred

    A quantum physicist would say, “100%, it is guaranteed to come up heads in one universe, and tails in another.”

  • Fred

    An inflationary cosmologist would say, “It is impossible to compute probabilities in the eternally inflating multiverse and so I can’t give you a proper answer. If I do give you an answer, rest assured it will lead to non-sensical paradoxes, which will allow me to write many new follow up papers.”

    Sean, speaking of such things, what are your thoughts on 1108.3080?

  • Tom of the Sweetwater Sea

    The coin has no memory. It does not know what it did last time.

  • STAN


  • Fred

    Actually Sean, a math student probably would say “We can’t predict the future from the past.” While a statistician, or a sensible physicist (such as an experimentalist) would be the ones saying “individual trials are uncorrelated.” Mathematicians are smart, but are notoriously bad at understanding how things work in the real world. They hate making approximations etc. Thats why they didn’t pursue science, but studied pure mathematics, and wouldn’t dare do something as imprecise as making a real world prediction.

  • HappyEvilSlosh

    A mathematician would say “It yet hasn’t been established the coin is fair”, where you proceed from there depends on whether you subscribe to the frequentist or bayesian school of thought. Uncorrelated trials generally make it easier, not more difficult, to calculate probabilities.
    (Also not all mathematicians are pure mathematicians)

    –a mathematician

  • Stu Savory

    Just FYI : a question I ask my students is
    “Given a single biased coin, how can you make fair tosses?”

    Fair meaning 50:50 outcomes within the bounds of probability theory.

  • Fred

    A creationist would say, “Probability is only a theory. God decides which side the coin comes up.”

  • socle

    I understand Greenstein’s point, but this mathematician would have agreed with the professional gambler, even as a student. I would think that pretty much any competent person would, tbh.

  • puzzled

    Stu: Make a pair of coin tosses. If the outcome is TT or HH, discard the pair and try again. If the outcome is TH, pronounce this to be tails; if it is HT, call this heads.

  • Cusp

    While each flip is independent, a Bayesian would use one to update their knowledge of the fairness of the coin.

    Knowing the past does eventually make a prediction about the future.

  • Brendon J. Brewer

    It depends on the prior information. I’d say in most situations the correct answer is the third one.

  • Jorge

    Bernanke would say: “Let’s just make more coins and see what happens”

  • Tatek

    I agree with Jorge. I would extend the argument and propose that we create a bigger world and then see what happens

  • Roland Kie

    A determinist would say ‘It is calculable what it would turn out to be according to how the person’s movement, the movement of the surrounding air, the weight and side of the coin before it was flipped, the moment when the person catches the coin etcetc’ but, of course since he won’t heve the power to calculate he wopn’t really have an answer and…then it goes nowhere…slow.

  • jt512

    A statistician wouldn’t use the word “uncorrelated” when the question is about independence. Hopefully, a mathematician who had taken a mathematical statistics course wouldn’t either.

  • Fred

    A statistician would use the word “uncorrelated” knowing that it obviously means the same as “independence” when the random variable is binary: heads or tails (although jt512 does not seem to know this trivial fact, even though jt512 claims to be such an expert on statistics and is so critical of other people’s comments).

  • BS vs ML

    What is really interesting is two different inductive processes competing.


    Black Swan vs Moore’s Law

  • Scott Aaronson

    A theoretical computer scientist might say: “choosing a prior distribution over possible sequences of coin flips where each sequence gets weighted by 2 to the minus its Kolmogorov complexity, and given the outcomes of the first ten flips, we find that ‘heads every time’ gets assigned an overwhelmingly large posterior probability; therefore, I agree with the professional gambler.”

    General Lesson: Any time we find that “math” disagrees with reality, the problem is never with “math”—it’s with us, for using the wrong math! :-)

  • Tony P

    We can do this – let’s check the math folks – a Bayesian would find the probability of heads to be 11/12?

  • Ken

    Don’t forget the gamer’s explanation. It combines the novice gambler’s naive understanding of probability with the profession gambler’s immediate “I can exploit this to my advantage”.

  • jt512

    Actually, the question isn’t about correlation or independence; it’s about what the probability of the next flip being heads is. However, if the probability of “heads” is 1 (as it appears that it may be), then the trials are still statistically independent, but their correlation is undefined. So, no, Fred, “uncorrelated” does not mean the same thing as “independent” if the trials are Bernoulli.

  • jt512

    @Tony. I concur: 11/12, using a uniform prior.

  • Austin Frisch

    As a former professional gambler, I’ve had the best of it after going all and lost 10 times in a row, on more than one occasion. Typically, I then proceeded to dig out some of my teeth with a grapefruit spoon.

  • Fred

    @jt512 23, one defines X and Y to be uncorrelated when the covariance Cov(X,Y)=E(XY)-E(X)E(Y)=0. Lets assign Heads=+1 and Tails=-1. Then in the case in which the probability of Heads is 1, then E(XY)=1 and E(X)=E(Y)=1, and therefore Cov(X,Y)=1-1*1=0, and so they are clearly uncorrelated. jt512 seems to be deeply confused about basic statistics; this would be fine, if not for the fact that jt512 is putting themself out there as some kind of expert and so critical of everyone else.

  • jt512

    Well, Fred, I guess I have to concede that you’re right here. Most references that I can find that define “uncorrelated” defines it in terms of the covariance, rather than the correlation coefficient.

    I don’t think, though, that that misunderstanding implies that I’m “deeply confused about basic statistics.”

  • Fred

    ok jt512, sorry if i seemed impolite.

  • JW Mason

    This is a really important point.

    In all kinds of thought experiments and logic puzzles (and in, like, every paper ever in moral philosophy), we’re told, “Assume you’re in situation X” and then asked to reason in some way from there. But the real world is never like that. Whatever information we have, we’re using not just to draw some conclusion conditional on being in situation X, but also to update our prior as to whether we really are in X.

    What makes it trickier is that in social contexts we all have some very strong priors that we can’t articulate clearly, or that we’re not even conscious of.

  • Ben

    In a strictly frequentist approach, you would say that based on the data, the maximum likelihood estimate of the probability of the coin coming up head is 1, so there is no reason whatsoever – purely based on the data – to bet on tails.

    In a Bayesian approach, you would look at the probability of the coin coming up head as a random variable, with is own prior distribution. The prior can be anything that describes your prior assumptions about the coin, before observing any flips. Some people may be very confident that the coin must be fair and they trust the coin flipper to be fair too – this would mean that their posterior belief, after observing 10 consecutive heads, would only shift by a little bit towards heads. Other priors are also possible.
    However, one thing is clear: unless your prior distribution has a bias towards TAILS, you will always infer that a HEAD is more likely on the 11th flip.

  • Tom Weidig

    I agree with Ben.

    You can take two views:

    a. to let yourself be guided by empirical data: in this case, bet on heads.

    b. to let yourself be guided by your theoretical understanding: I have no clear evidence to suggest that the coin is not fair, so equal probability. If I have, I will act according to a.

    This hinges on what “no clear evidence is”. If you define it as no falsifying evidence, heads coud come up a millions time and you don’t change your stance, because it is not impossible for this to happen with a fair coin. But if a close analysis of the physical coin suggest a physical difference, this will falsify the hypothesis “the coin is not fair” and you act according to a.

  • Sam Cox

    Probability is an integral part of a 4D cross-section of the universe- right down to quantum mechanics! The fact that nothing is completely certain or absolutely impossible, however offers a clue to the possibility that what happens in reality is just about completely certain. The chance that the sun will rise tomorrow appoaches 1, yet there was a time when the sun did not rise and there will be, in the future, a time when the sun no longer rises. My existence was next to impossible-almost 0- but here I am!

    Each time a coin flips, so long as great pains are taken to keep the flipping process random, the chance of heads vs tails is exactly the same. However I remind my students a coin has three sides, not two and the chance the coin will stand on end increases as the thickness of the coin increases. If a coin is thick enough and becomes a cylinder, landing on the ends becomes very improbable- and landing on its third side becomes almost certain.

    Seen from its proper perspective….its complete perspective, the universe much more clsely approximates a cylinder of vast length than a coin….

  • Sam Cox

    A few additional thoughts….

    1. The chance that a cylinder of vast length would stand on either end approaches zero
    2 The chance that the cylinder will remain on its third surface approaches one.
    3. The two ends are plane surfaces
    4. The third surface is circular
    5. The lower dimensional similarity to Euclidean SRT….local flat space and GR with its marginally closed spheroid geometry.
    6. The overall stability of the sytem
    7. The possbility of motion and change as the cylinder rolls in a certain direction…..

  • Phillip Helbig

    “A professional gambler would say, “There must be something wrong with the coin or the way it is being flipped. I wouldn’t bet with the guy flipping it, but I’d bet someone else that heads will come up again.””

    Jimmy the Greek said that he could make money from gambling in Vegas because he never played the games; he would do things like “I bet that blackjack player will bust on the next round” and make bets on the outcomes of the games.

  • Robert Garisto

    It depends on what your prior probability is for the hypothesis, ‘the coin tosses are all fair’ (depending on whatever other information you would use if this were a real world situation). If, for whatever reason, you take that prior probability very very close to 1 (say because you’d tested the coin yourself), then you’d predict the heads-on-next-flip probability to be very close to 1/2. The professional gambler is a cynic and assigns a nonzero probability to unfairness somewhere. She has the right strategy provided it’s an even money bet. She would be a fool to pay odds to someone though, unless she had evidence of unfairness. Put it this way, the odds of 10 heads in a row is 1024:1, but far fewer than 1 coin in 1000 is unfair. So a random coin which results in 10 heads in a row is still likely to be fair. (The most likely explanation for a random coin coming up heads 10 times is that the coin is fair and there was a statistical fluctuation.)

    But if it is 100 heads in a row, Hamlet is having you killed.


Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Cosmic Variance

Random samplings from a universe of ideas.

About Sean Carroll

Sean Carroll is a Senior Research Associate in the Department of Physics at the California Institute of Technology. His research interests include theoretical aspects of cosmology, field theory, and gravitation. His most recent book is The Particle at the End of the Universe, about the Large Hadron Collider and the search for the Higgs boson. Here are some of his favorite blog posts, home page, and email: carroll [at] .


See More

Collapse bottom bar