George Will: Time For Some Significant Fact-Checking

By Carl Zimmer | February 22, 2010 12:59 am

A year ago this month, George Will wrote a howler of a column in the Washington Post about global warming, loaded with scientific errors and profoundly illogical arguments. It would not have survived even the most perfunctory fact-checking–despite claims from the Washington Post that his columns go through a “multi-layered fact checking process.” In subsequent months, Will has continued to offer new climate howlers, and this Sunday he provided us all with a dubious one-year birthday gift.

In Will’s latest piece, he yet again declares global warming a construction of hysterical climate scientists who, in his words, “compound their delusions of intellectual adequacy with messiah complexes.” This time, he claims that climate scientists themselves are finally confessing that it’s all been a whole lot of hooey.

Will backs up this claim with a link to a BBC interview last week with Phil Jones of the University of East Anglia. A BBC journalist asked Jones questions, some of which had been submitted by unnamed climate skeptics, including this one:

Q: Do you agree that from 1995 to the present there has been no statistically-significant global warming?

A: Yes, but only just. I also calculated the trend for the period 1995 to 2009. This trend (0.12C per decade) is positive, but not significant at the 95% significance level. The positive trend is quite close to the significance level. Achieving statistical significance in scientific terms is much more likely for longer periods, and much less likely for shorter periods.

This statement then got run through a sausage grinder run by journalists who are apparently both innumerate and illiterate. The Daily Mail declared,

“This week the unit’s former head Professor Phil Jones, performed a majot [sic] u-turn and admitted there had been no ‘statistically significant’ global warming in the last 15 years.”

This version of the story, which makes Jones sound like he was making a confession under enhanced interrogation techniques, ended up on the Wall Street Journal editorial page and today in George Will’s column:

Global warming skeptics, too, have erred. They have said there has been no statistically significant warming for 10 years. Phil Jones, former director of Britain’s Climatic Research Unit, source of the leaked documents, admits it has been 15 years.

Will doesn’t tell us exactly who these skeptics are who claimed there had been no “statistically significant warming” for 10 years. I have no way of knowing if they in fact exist. Will himself has been loudly beating the “no-warming-for-a-long-time” drum over the past year. But he has backed up this claim simply by searching for the hottest single year in recent history. “According to statistics published by the World Meteorological Organization, there has not been a warmer year on record than 1998,” he wrote in April. Will continued to claim that global warming has stopped since 1998 even after the secretary general of the World Meteorological Organization wrote into the Post to explain why Will was wrong.

In his latest column, Will added the fancy, shiny new term “statistically significant” to his claim that there has been no global warming. But in doing so, he misleads his readers about what statistical significance actually means.

To see why, take a look at this graph. NASA scientists have been building it for years now, using weather records from around the world. Other graphs built by other teams of scientists have produced similar patterns. If you only look at a small vertical slice of the graph, you’ll see the temperature jump up and down and up again. That’s the sort of pattern you’d expect from a system as big and noisy as the planet’s climate. There are lots of sources of variations in the average global temperature, such as El Nino, a natural oscillation in the movement of heat in the oceans.

Sometimes these hopping temperatures don’t seem to go anywhere in particular. In other cases, there appear to be trends lurking under the noise. To test a hypothesis like this, scientists estimate how likely it would be for an apparent trend to be nothing more than the noise in the climate system. They then set a threshold for those odds.

In many branches of science, researchers set that threshold at 5%. In other words, if there’s only a 5% chance that a particular pattern of temperatures was the result of pure noise, scientists will call the trend “statistically significant.” If, on the other hand, the probability turns out to be 5.1%, the trend is still likely not to be the result of noise, but it’s not officially statistically significant.

“The boundary of .05 should be seen as a guide to interpretation, not as a clear boundary between truth and fiction,” Michael Whitlock and Dolph Schluter write in their book, The Analysis of Biological Data.

Just because a trend over a particular stretch of time doesn’t quite meet the 5% cutoff doesn’t necessarily mean it’s not real. It just means that scientists cannot reject the null hypothesis that noise is the cause. One way scientists can deal with this challenge is to look at longer sets of records. In the case of climate, looking at longer stretches of time reveals that there is indeed a real trend of warming temperatures. Just because the BBC’s questioner arbitrarily set the cutoff for analyzing the climate at 1995 doesn’t change that fact. Jones openly addressed this fact, but George Will conveniently omitted it.

Significance is one of the basic concepts of statistics that everybody should learn about. We rely on these concepts to judge not just the state of the climate, but also the meaning of clinical trials of drugs, the conclusions of psychology experiments that help reveal the inner workings of the mind, and all manner of other discoveries. In today’s column, George Will isn’t just making misleading statements in the service of trying to foster doubt about climate change. He’s also helping to muddle our collective scientific literacy. Why the editors of the Washington Post’s editorial page want to be a party to this is a mystery to me.

[See Skeptical Science and Tamino for more.]

Comments (29)

Links to this Post

  1. westindianplanet.com | February 23, 2010
  2. lathropcity.com | February 23, 2010
  1. RealTH

    The jump probably was when they went from 6000 temp readings to 1500, eliminating all the colder regions.

  2. chris johnson

    Hello Carl

    There is another point that is often lost in discussions of the statistical significance of climate trends. This is that conventional statistical tests based on regression or correlation are not correct for these data in the first place, because they assume that the temperature in each year is independent of the temperature in previous years.

    But, of course, that’s not true. There is a lot of inertia in the climate system, and cyclic processes that give strong correlations in temperature from year to year. If we used standard regression techniques, it should be because they give as a convenient way to estimate the trend in the data, not because they prove that the trend is meaningful or otherwise. For that, we need to look at mechanisms, and try to decide if the ones that we know about can explain the magnitude of the trend. That is what most real climate science is about. Arguing about statistical significance at the 5% level is a silly distraction.

    chris

  3. Wil

    For heaven’s sake, settle down Carl, or you might burst a vein.

    The zealous tone in the above article makes it hard to take it seriously. You must remember that AGW is not a religion. At least it is not supposed to be.

    I can not see how you can put any faith in data and graphs from scientists and organizations who have been caught knowingly lying, cherry picking data, deleting data, selectively sampling, calling idle speculation as proven fact, and so on.

    How many dozens of times does your local used car dealer have to lie to your face, before you start to doubt him?

    [CZ: Wil, what does "tone" have to do with statistical significance? If I rescue a puppy from a well and then tell you that the moon is made of green cheese, it's still not true.]

  4. Colleges need to start offering classes for scientists on how to convey their messages to laypersons, so that they can learn to explain things without assuming that everyone knows what significance levels and so forth are. Sure, the journalists’ should seek to understand what these things mean and convey it to us without spin, but they don’t appear to be doing very well at that, and of course columnists like George Will who have an entrenched ideological opinion are only interested in how they can spin things.

    “Achieving statistical significance in scientific terms is much more likely for longer periods, and much less likely for shorter periods.” Is everyone supposed to know what that means? If only Jones had just explained that global temperatures vary considerably from year to year and so it’s hard to prove a trend from only 15 years’ worth of data (and even harder to disprove it, given the actual data). People might still not fully understand, people might still deny it, but at least he wouldn’t be misleadingly quoted by newspaper men and ideologues. Know thy audience!

  5. Toad

    To Wil (#3):

    Please point to the location in the article that has a “zealous tone.”

    And, pray tell, what exactly would motivate a climatologist to “lie to your face?” An ironic question for you to ask, seeing as the article you didn’t read discusses George Will’s convenient omission of the term “statistical significance” to misrepresent the words of a climate scientist.

  6. Katharine

    Wil, consider the zealous tone from your side.

    I don’t see how you put MORE faith in your side, which has no data to show.

    How many dozens of times do you have to trust total bull over actual data?

  7. FastEddie

    Wil, let me get this straight. You rightly mock certain scientists for lying and cherry-picking data, yet when Carl points out this same fault with George Will you mock him?

  8. kharris

    Funny that “Wil” (sounds like…) relies so heavily on tone and so little on substance to criticize Zimmer for a “zealous tone”. Odd also that Wil hears a zealous tone coming through in a description of the use of statistical analysis. Perhaps it should be mentioned that “statistical significance” has a very precise meaning and that it is often misused in ways that are quite misleading. An insistence on strict correctness in drawing inference from statistical analysis, among those who understand statistics, is not the approach of a zealot. It is, rather, the approach of someone who cares about drawing the right inference, rather than the wrong one.

    The analogy to a used car dealer is, it should be pointed out, simply not apt. A single used car dealer’s tendency to lie should be used to assess the statements of that dealer, but tells us nothing about other merchants. There have been isolated cases of climate scientists doing some of the things that Wil cites, and those scientists need to be doubted in their other claims. There are, of course, plenty of global warming skeptics who have done all the things Wil mentions, apparently Wil among them. It is simply dishonest to pretend that the global warming hypothesis is based on jiggered data and deceptive analysis. The hypothesis is sufficiently well founded that the deviations from good science that have been uncovered don’t really do much to harm its credibility.

  9. James Hathaway

    Sigh. OK, he’s a columnist, which means that he doesn’t actually have to pay attention to context of quotes or to the data and what they mean, since all he is writing is basically “opinion,” which is, of course, completely up to him. If he wants to say that they sky is not blue, but just a very energetic shade of red, well, that’s his right to “interpretation.” What I find more alarming about the current state of how the broad public decides something is accurate or not is the view that everything is really relative to our political perspective (as in the previous comment here), and if we don’t trust one source (for whatever reason, but primarily because we don’t agree with them politically) it is perfectly acceptable to trust an opposing source with no critical analysis. In other words, a lot of people have completely given up on standard ways of determining what we used to call “objective” truth, and have decided that “it’s all relative” — especially to political viewpoint. For me, this is terribly ironic because I can remember a decade or so back when all the conservatives were railing against “Godless relativism.” Philosophically, they are chameleons. You’re right — it will be entertaining to see if the Washington Post claims they fact-checked this one too, since verification would require not only having very loose standards on interpreting context but also a general disregard for the meaning of statistics. One wonders whether or not fact checkers are required to understand statistics.

  10. Michael Heath

    While we should continue to call-out how Mr. Will continues to misrepresent both science, scientists, and AGW, the larger question is why the WaPo is willing to knowingly publish a column whose conclusions are so obviously dishonest. It also seems like smaller newspapers who purchase nationally syndicated columns like Mr. Will’s, which is where most people are exposed to him, would start to demand some quality control from the companies from which they procure such columns.

    My small local newspaper has reacted by dedicating 2/3 of one page in their Saturday editorial section to a FactCheck.org column covering one giant newsworthy lie from earlier in the week. For example, a week ago last Saturday they compared the honesty of VP Biden’s comments regarding national security vs. ex-VP Cheney’s comments from his interview on ABC’s Sunday morning newscast.

    My newspaper has also been gracious enough to publish subsequent columns by myself or others like me that are up to speed on the science where we correct the record regarding the general denialism claims we encounter with Mr. Will along with certain events like the so-called Climategate disaster which commenter Wil and so many others grossly misrepresent as refuting the legitimacy of the evidence and theory. However all that does is create a ‘he said / she said’ scenario in readers’ minds, which is giant fallacy of balance defect though better than people falsely believing that AGW is not peer-accepted or tenuously held by climate scientists.

    What we really need are publishers and editors who have a near-zero tolerance for either promoting false assertions or making arguments dependent on false assertions. However it now appears this capability is both eroding or in Fox News’ case, actually increasing loyalty to their brand.

  11. david

    Carl, Did the quotes below give you the inspiration to post about Global Warming? By commenting on it I was merely trying to point out what those who oppose the macro-evolutionary viewpoint have known for years!

    “He said that the controversial e-mail exchanges about climate change data had caused people to suspect that scientists “oppressed free speech”.

    His fellow panel members, including Lord Martin Rees, president of the UK’s Royal Society, agreed that scientists needed to be more open about their findings.

    “There is some evidence that the distrust has spread,” Dr Cicerone told BBC News. “There is a feeling that scientists are suppressing dissent, stifling their competitors through conspiracies.”

    “Public opinion polls are showing that the answers to questions like: ‘how much do you respect scientists?’ or ‘are they behaving in disinterested ways?’, have deteriorated in the last few months.”

    He said that this crisis of public confidence should be a wake-up call for researchers, and that the world had now “entered an era in which people expected more transparency”.

    “People expect us to do things more in the public light and we just have to get used to that,” he said. “Just as science itself improves and self-corrects, I think our processes have to improve and self-correct.”

    [CZ: david--I'm guessing that these quotes are from the recent AAAS meeting. I was at the meeting, but did not get to that particular panel. It did not give me the inspiration for this post. As you can see if you follow the links, I've been following Will's trail of errors for a year now.]

  12. gribley

    Much as Will is twisting things, as is his wont (as you have shown), I think Jones gets some of the blame for poor communication. I largely agree with Matunos; we need more climate scientists who are trained to communicate their results clearly to the media. The interview question you quote is based on several false premises: That a brief window of time is an appropriate; that 1995 is a good time to start measuring; that statistical significance tells us anything useful. It’s a trap, and by answering “Yes, but…” Jones fell smack into it.

    Chris # 2 — A good point, but there are ways of analyzing correlated data, and I assume that one of these was used in this sort of calculation. Otherwise you could never report on the statistical significance of, for example, weight gain or changes in blood cholesterol. That said, statistical significance testing is pretty generally useless.

  13. Jones was simply referring to the fact that with a dataset as noisy as global temperature, you need to look at a time series of 15 years or longer before you start hitting a 95 percent confidence level. With a shorter time series (say 2000-2009) there’s too much noise and not enough signal to say the trend statistically significant.

    The climate blog “Open Mind” gives a useful tutorial on this subject:

    http://tamino.wordpress.com/2009/12/15/how-long/

  14. Robert the Red

    If you get the data from the chart above, and do a simple t-test to see if the 1995-2001 data averages smaller than 2002-2009 (say), you’ll find that the difference is quite “statistically significant”. This effort takes about 2 minutes.

  15. Katharine

    It’s WaPo. I don’t expect the local rag to be unbiased.

    This raises the notion that everyone should be made to take a statistics class. At least, science majors. I don’t know why undergraduate science majors aren’t required to take stats.

    [CZ: I was lucky enough to go to a public high school that offered a stats class. It's served me better than trigonometry, that's for sure!]

  16. Katharine

    I would add that the public had better get conditioned to be more passionate consumers of knowledge. Toss that American Idol @$%^%#$.

  17. Katharine is correct. Even as someone who stinks at math, and has never taken a statistics course, I have had enough self-education to get the gist of confidence levels, the meaning of statistical significance, and how you go about testing for it. For scientists, these concepts are so deeply embedded into the DNA of their work that they sometimes forget that lots of people have no clue what the term “statistical significance” actually means.

    In George Will’s case, he cannot plead ignorance since he has plenty of time to research a column and has in his employ a “researcher” to supposedly do all this fact-checking stuff on his behalf before he writes the first sentence of a column. And there’s always the phone, where he can call up Phil Jones and interview him. But that requires actual effort.

    As Carl notes, George Will has already been publicly called out for grossly misinterpreting and misstating basic scientific data and concepts on this same topic, so the only likely motive here is the standard established by the U.S. Supreme Court in the 1964 libel case Times v. Sullivan: “actual malice or reckless disregard for the truth.”

  18. Pericles

    Carl,

    The article you wrote included a graph titled “Global Land-Ocean Temperature Index,” which purported to show temperature anomalies from 1880 to the present. After I looked at it, I had a few questions about it:

    1. Where in the world were these readings taken? That is, which countries were they taken from? Which areas of the world? Were they taken from cities, rural areas, or a combination of them? How have the areas where they have been taken changed since 1880?
    2. That graph shows a temperature anomaly, but what is the anomaly based on? Average high temperatures? Average low temperatures? Average temperatures?
    3. Which time period is the baseline?
    4. Have the number of reporting stations changed at all between 1880 and today?

    Answers to these questions would make that graph easier to understand.


    “Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: ‘There are three kinds of lies: lies, damned lies, and statistics.’” ~Mark Twain, autobiography, 1904

    [CZ: The papers from which this graph come from are supplied on the web site.]

  19. To Wil #3: “Wil” of course is amusing when we consider focus here is “Will”. So, based on the standard you provide, we assume that you do not listen to George Will at all since he has been caught in so many (repeated) misrepresentations and outright errors, without any corresponding record of correcting them.

  20. Pericles –

    1. NASA-GISS has all of the historical temp. data available on-line from their servers as well as the source code for all of the predictive climate models they use. Just google it.

    2. All the questions you are asking have been extensively analyzed and parsed many many times by scientists at NASA-GISS.

    3. The Mark Twain quote seems to be, in the context of your questions, an argument by ignorance or personal incredulity. Twain’s quote is frequently misused as an ad hominem against the presentation of data in statistical form. He did not mean this, of course. He meant the opposite, that represent of data in the medium of statistics is too often assumed a priori to be authoritative simply because it is displayed graphically or quantitatively. He did not mean the corollary, which you seem to imply, that any data presented graphically or quantitatively must be false because it is possible to use these display formats to displace false information. That’s like saying you can’t trust the telephone because one time you got a call from a telemarketer.

  21. ianam

    Why the editors of the Washington Post’s editorial page want to be a party to this is a mystery to me.

    They are ideologues with vested interests.

    Also, would it be “zealous” to note that Wil is a ignorant, dumb, hypocritical anus? That he “can’t see how” rational people can look past his ad hominem mischaracterization of climate scientists to the data does not speak in his favor.

    I think Jones gets some of the blame for poor communication.

    Nonsense. It’s a single limited response to a specific question, yet he still manages to mention that the trend is positive, it’s close to the 95% significance level, and reaching such levels is less likely for such short intervals.

  22. SLC

    Of course, anyone looking at the graph will readily discern that the reason for the apparent lack of significance is the anomalous result for 1998. Clearly, 1998 is an outlier. Remove it and I will guarantee that the significance level will greatly exceed 95%.

  23. 1998 is not an outlier. Based on NASA-GISS global temperature data, which unlike Jones’ HADCRU dataset, covers the Arctic, 2005 is the warmest year on record. 2009 is the second warmest.

    Do I need to do linky?

    I guess I do:

    http://www.realclimate.org/index.php/archives/2010/01/2009-temperatures-by-jim-hansen/

  24. For those too impatient to do the linky, it says:

    “The past year, 2009, tied as the second warmest year in the 130 years of global instrumental temperature records, in the surface temperature analysis of the NASA Goddard Institute for Space Studies (GISS). The Southern Hemisphere set a record as the warmest year for that half of the world. Global mean temperature, as shown in Figure 1a, was 0.57°C (1.0°F) warmer than climatology (the 1951-1980 base period). Southern Hemisphere mean temperature, as shown in Figure 1b, was 0.49°C (0.88°F) warmer than in the period of climatology. The global record warm year, in the period of near-global instrumental measurements (since the late 1800s), was 2005.”

    Source:
    http://www.realclimate.org/index.php/archives/2010/01/2009-temperatures-by-jim-hansen/

  25. SLC

    Re Doug Watts

    Excuse me, 1998 was an outlier at the time it was taken, as it was well above the adjacent data points.

  26. Daniel J. Andrews

    1998 wasn’t the warmest year as George asserts. Everyone knows it was 1934. You can’t trust those global temperature records, those devious scientists have altered them.

    Besides it was warmer in the 40s and in the medieval warm period than it is now. And warming did stop in 1998. Just check the … uhhh…. global .. temperature recor….rds. Aw nuts.

    Ok, if global warming stopped in 1998 then how does George explain Mars and Pluto are warming? :)

    More seriously, I rather like that one. The sun, which is apparently creating warming on Mars and even distant Pluto, is also causing the Earth to cool or not heat up at all since 1998? There’s a Nobel Prize waiting to be won somewhere if that can be explained.

    So many contradictions, so little time. Earth is naturally warming, the Earth isn’t warming, the Earth is actually cooling. It’s the sun, it’s cosmic rays, it’s orbital inclination, its undetectable underwater volcanoes, it’s El Nino, all doing the warming—unless it isn’t warming or it is actually cooling in which case it isn’t the sun, it isn’t…..etc.

    I think at this point we have to conclude George is a bit thick, or his logic and critical reasoning centres have eloped off together, or he’s lying. Hard to think of another alternative (a brain tumour maybe?).

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

The Loom

A blog about life, past and future. Written by DISCOVER contributing editor and columnist Carl Zimmer.

About Carl Zimmer

Carl Zimmer writes about science regularly for The New York Times and magazines such as DISCOVER, which also hosts his blog, The LoomHe is the author of 12 books, the most recent of which is Science Ink: Tattoos of the Science Obsessed.

ADVERTISEMENT

See More

ADVERTISEMENT
Collapse bottom bar
+

Login to your Account

X
E-mail address:
Password:
Remember me
Forgot your password?
No problem. Click here to have it e-mailed to you.

Not Registered Yet?

Register now for FREE. Registration only takes a few minutes to complete. Register now »