Science is Hard

By Sean Carroll | April 8, 2011 9:46 am

The 3-sigma bump reported by Fermilab on Wednesday has garnered a lot of attention. Understandable, since it might be a precious sign of particle physics beyond the Standard Model — but it’s also just a 3-sigma bump, and usually those go away.

Via Matt Strassler and Lisa Randall, here’s a set of plots that helps indicate exactly how hard this game really is. The plots can all be found on this web page at CERN. In comments Matt and Peter note that they were made by Tommaso Tabarelli de Fatis, as explained on Tommaso Dorigo’s blog. Here is the original plot from the CDF paper:

We’re looking at the number of events that produce a W boson and two jets, as a function of the energy of the jets. The bottom plot is all the data, while at the top they’ve subtracted off most of the Standard Model background, leaving only the predicted red curve from WW/WZ events. You see the extra little bump around 150 GeV, that’s what’s getting everyone so excited. It’s unlikely that the data are a good fit to the prediction; the “KS (Kolmogorov-Smirnov) probability” is given as 5×10-5, which means that it’s not bloody likely.

But, just for giggles, let’s imagine that the energy of the jets wasn’t measured very accurately. Obviously the experimenter worked hard to get it right, and I would trust their judgment over my own any day, but you never know. Jets are complicated things with many particles in them, you can imagine being off by a bit.

So here is the same plot, just scaling the jet energy by two percent:

You see that makes a lot of the excitement go away. The KS probability is now 9×10-2, which essentially means … the bump has gone away. If we scale by 4%, it goes away almost entirely:

Now the data fit the Standard Model very well. If we keep cranking up the supposed error, we stop fitting again, because the data at very low energies begin to go astray:

There is an animated version of the plot which is fun to look at. Nobody is saying (I don’t think) that this is certainly what’s going on; it’s just an illustration of how difficult this game really is, and why people shouldn’t get too excited about three-sigma events. A little bit of excitement is good, and descending deep into cynicism is bad, but patience is really warranted. Let’s collect more data and see what happens.

CATEGORIZED UNDER: Science, Top Posts
  • http://www.math.columbia.edu/~woit/blog Peter Woit

    Tommaso Dorigo has an informative blog posting up today about this, see

    http://www.science20.com/quantum_diaries_survivor/blog/jet_energy_scale_explanation_cdf_signal-77886

    He explains that the plots are work of Tommaso Tabarelli de Fatis.

  • Matt

    The plots were made by Tommaso Tabarelli de Fatis, I first saw them on Tommaso Dorigo’s blog

    Update: I see Peter Woit beat me to it

  • http://brokenglassware.blogspot.com Scientist Charro

    A little bit of excitement is good, and descending deep into cynicism is bad, but patience is really warranted. Let’s collect more data and see what happens.

    Then why make such a big deal at this stage?

    Has anyone ever thought about the possible negative effects from publicizing something so much to later say that it’s not true? If someone wants to hype scientific results up, it should be done after it’s been confirmed.

    And, why does it happen a lot more often with particle physics?

  • Joey

    Do the authors of the plot explain whether one is allowed to shift the energy by 4% arbitrarily?

  • http://catastrophicforgetting.blogspot.com/ T.

    Thanks Sean (I think you actually have four times the same plot though)

  • http://blogs.discovermagazine.com/cosmicvariance/sean/ Sean

    Peter and Matt, thanks for the find; I’ve updated the post.

    T– look more carefully!

  • David

    Quoted from the article:
    “But, just for giggles, let’s imagine that the energy of the jets wasn’t measured very accurately. Obviously the experimenter worked hard to get it right, and I would trust their judgment over my own any day, but you never know. Jets are complicated things with many particles in them, you can imagine being off by a bit.”

    Sean,

    How accurate is the measurement of jet energy in a normal experiment? Does a difference of 4% in that measurement happen often enough to casually suggest it as an alternative explanation in this case?

    If 4% errors in measurement are common, then would you agree that the “excitement” generated by the report of this data is more of a media exercise than it is an indication of some new physics?

    I am very eager to hear what DZero comes back with, even if it confirms nothing more than a statistical fluke.

  • spaceman

    Sean,

    If 3-sigma results in particle physics usually go away, then why has this latest result gotten so much attention? In my opinion, it seems that we are entering a period of time in which it is truth or consequences for many of our cherished theories, as no gravitational waves have been detected, no dark matter has been directly detected, and the Higgs boson remains unfound as well.

  • http://thecosmist.com The Cosmist

    My personal theory is that the simplest explanation for this universe is that we are living in a simulation that makes up the rules on the fly and generates new regions of spacetime as we explore them. If there is a “Great Programmer” of this simulation I’m sure He is having quite a laugh at our attempts to make sense of His algorithms! ;)

  • Anonymous_Snowboarder

    @charro: I’m sure ‘hype’ happens in other fields too but that hep gets mainstream press while many other fields do not. In any event, I have no issue with someone putting up 3 sigma preliminary results or those which will require a more extensive search by other researchers. And certainly why shouldn’t experimentalists get some slack when theorists are always churning stuff out which is not five sigma and having it hyped?

  • Pingback: 8 April 2011 pm « blueollie

  • Also David

    @ #7 (David)

    Last I heard the uncertainty in the jet energy scale at CDF was about 3%, which seems enough to reconcile this difference but I would have expected the author to include this systematic before presenting this bump as 3-sigma. There are other ways you could accidentally find a bump too, if the Monte Carlo prediction of the background was normalized incorrectly (through no fault of the author) it might skew the background subtracted result. Basically there are a lot of things that *might* cause such a bump and the best way to test it will be more data.

    Tellingly, CDF didn’t see this bump in a similar analysis with leptons implying that any new particle would have to be very lepton shy. D0 didn’t see anything either but I suspect they’re going over all their data again. The LHC will hopefully be ready to release some results on a similar analysis by the end of the year, a lot of people will be very disappointed if CDF pipped them to the post.

    While we’re on the subject I saw a very good article written at a layman’s level over at io9:
    http://io9.com/#!5789759/has-fermilab-really-discovered-an-entirely-new-subatomic-particle
    Apologies for the hashbang

  • John

    Your point is valid, but I hardly think it’s fair (beyond being illustrative) to apply a rote scaling to every point there. It means not only did they get the jet energy wrong by 4%, but *in every bin* of the bump.

    Put differently, if we apply the same type of shift to the standard model prediction near 90 GEV, we can make it go away as well.

    I am *not* saying that the blip is real, nor am I saying it’s energy miscalculations. I’m just pointing out that shifting *every* bin is a tad misleading….

  • Pingback: New physics on Tevatron? « Symmetry

  • Pingback: Nachrichten aus der extremen Physik kompakt « Skyweek Zwei Punkt Null

  • chris

    thats what happens when funding runns out on a big experiment

  • chris

    you do have a charming way of saying they screwed up the jet energy measurment ^_^

  • a_ray_in_dilbert_space

    Although it’s probably going to vanish like dreams in the dawn, this result has generated interest precisely because it is the first indication of something going on beyond the standard model . It would represent fundamentally new physics.

    And this is going to be as good as it gets as the Tevatron is now a memory.

  • Pingback: Anomaly at the Tevatron Might Be Something Real? | Cosmic Variance | Discover Magazine

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Cosmic Variance

Random samplings from a universe of ideas.

About Sean Carroll

Sean Carroll is a Senior Research Associate in the Department of Physics at the California Institute of Technology. His research interests include theoretical aspects of cosmology, field theory, and gravitation. His most recent book is The Particle at the End of the Universe, about the Large Hadron Collider and the search for the Higgs boson. Here are some of his favorite blog posts, home page, and email: carroll [at] cosmicvariance.com .

ADVERTISEMENT

See More

ADVERTISEMENT
Collapse bottom bar
+

Login to your Account

X
E-mail address:
Password:
Remember me
Forgot your password?
No problem. Click here to have it e-mailed to you.

Not Registered Yet?

Register now for FREE. Registration only takes a few minutes to complete. Register now »