“You know, the most amazing thing happened to me tonight. I was coming here, on the way to the lecture, and I came in through the parking lot. And you won’t believe what happened. I saw a car with the license plate ARW 357. Can you imagine? Of all the millions of license plates in the state, what was the chance that I would see that particular one tonight? Amazing!” — Six Easy Pieces

The winner of the toss gets to DECIDE whether to kick or receive.

Despite what people say about gifts, NFL coaches usually find it more blessed to receive.

The Sigma for Higgs taken in context of the Bayesian analysis of what the pre-test expectations for the experiment are, and accounting for 2 separate experiences coinciding makes the chance of the phenomenon being real much higher than simply saying that they are likely to be wrong 1/1000 times, and that kind of error is pretty common.

By the way, same goes for the superluminal neutrinos. There are more than one set of experiments revealing the same result, which raises the Bayesian chance of it being a true finding. That does not mean that they beat the speed of light, only that they traverse known space faster than photons can. There may be other reasons besides speed that they do that (like unknown space shortcuts only they have access to). Either way, the street would say the odds of the findings being real are higher than a single result would indicate.

Three-sigma results will be observed in about 3 draws out of 1,000 from a normal distribution. So if you form a null hypothesis about a particular data-generating process (DGP) (e.g. a fair coin is being flipped), draw a sample from it, and observe a three-sigma departure from the predicted value, then if your prior probability for the alternative to the null (that it is not a fair coin toss, in this case) was not substantially less than one in 300, then you should seriously consider rejecting the null and accepting the alternative. Most of the comments here are various suggestions about what rejecting the null here would mean concretely.

But! This is not the situation described by this post. We did NOT first form a hypothesis about the DGP and then draw a sample from it. Rather, we are only talking about coin flips because we *already* observed the anomalous result. So the relevant question is, given the universe of comparably salient DGPs (in the Super Bowl, in professional sports, among stuff going on this past weekend, among stuff Sean might plausibly blog about — whatever we decide the relevant universe is) what is the probability of a three-sigma result being observed among at least one of them? And given that, for any reasonable definition of the potential universe of DGPs, there are far more than 300 of them, the answer is going to be close to 100%.

Sean gets this, obviously — the post was clearly tongue in cheek. But it seems that many commenters here don’t.

As for the application to physics, that’s way above my pay grade. But it depends whether (or to what extent) the sample with the three-sigma anomaly is drawn from a DGP identified as of interest on prior theoretical grounds, or whether it’s the result of a fishing expedition. Rich T.’s link suggests it’s often the latter; I don’t have any idea if that’s right, but it’s the right question.

See http://cms.web.cern.ch/news/should-you-get-excited-your-data-let-look-elsewhere-effect-decide

Not only might one ask what are the odds that it will happen in the history of the Superbowl instead of the last 14 years (prior to this year), but what are the odds that such a string of coin flips would happen in a series of high-profile games. It would be just as noticed if it happened at the national-championship game for college football. It would also be noticed if a single team won (or lost) 14 coin tosses in a row. Of course we also pay attention to many other sports stats as well.

So the odds that such a coincidence will astound us are actually quite good — indeed all but certain.

I assume that the particle physics guys at the LHC are not just looking any old statistical anomaly in their millions of data points as they WILL find one. Rather the theory must say what it expects to find. When they calculate the sigma for the proposed Higgs observations, are they calculating the odds of finding anywhere in the proposed range of energies or just the odds it will be found where it was found? I hope it is the former. But even if it is not, the vast majority of statistical quirks that one one might dig out of the LHC data are not what Higgs hypothesis predicts. There is a big difference between noticing a statistical quirk and predicting one in advance.

The 14 coins did not produce the same outcome year after year, they just resulted in 14 times one winning the outcome of the flip whatever it may have been by virtue of either guessing or of the other team guessing incorrectly. Which brings me to my point:
There is the overlooked random event: which team gets to make the call for the outcome.

“Statistics are like bikinis. What they reveal is suggestive, but what they conceal is vital.”

—Aaron Levenstein

There may be 16 correct calls in a row, but there is only one lucky streak!