#112,
“This is flat out wrong. Peer reviewers absolutely do check calculations, check references and check observations.”

When Steve McIntyre was reviewing a paper, he asked the journal to provide the data so he could check it. The journal editor (Stephen Schneider I believe) replied that nobody had ever asked for data before, and that he would not be provided with it.

When he asked for background data on a paper during his review for the IPCC, Susan Solomon not only refused, but said that he would be thrown off the list of reviewers if he “abused” his position as an IPCC reviewer by demanding data again.

No. They don’t check. Until McIntyre came along, nobody had checked Mann’s data – if they had, they would immediately have found all the flaws in it. They would have immediately found that the result wasn’t calculated in the way Mann said it had; that there were a whole list of unpublished steps.

It may be the practice in other areas of science (it depends on the journal, I suspect), but there are plenty of other academics who have said exactly the same thing. Peer-review does not offer any guarantee of accuracy.

Steve McIntyre has said in the past that he’s inclined to accept basic AGW theory, and that his view is that politicians would be sensible to accept the IPCC’s findings. For those who think it’s a matter of politics, he has also described his political position as corresponding to a “Clinton Democrat”. Canada tends to be a bit more liberal than the US, anyway.

But if bias against AGW renders work untrustworthy, then what does bias for AGW do?

NASA do indeed have a lot of climate change information, but unfortunately, while some bits of NASA’s output are bullet proof, other bits aren’t. In particular I have indeed seen some of the code they used, and even as a person inclined to doubt their output, I was shocked.

Here’s a bit:
SUBROUTINE CMBINE (AVG,WT,IWT, DNEW,NF1,NL1,WT1, ID,NSM,NCOM)
C****
C**** Bias of new data is removed by subtracting the difference
C**** over the common domain. Then the new data are averaged in.
C****
COMMON/LIMIT/XBAD,NLAP
DIMENSION AVG(*),DNEW(*),WT(*),IWT(*)
C**** Loop over years
NSM=0
C**** Find means over common domain to compute bias
SUMN=0
SUM=0
NCOM=0
DO 10 N=NF1,NL1
IF(AVG(N).GE.XBAD.OR.DNEW(N).GE.XBAD) GO TO 10
NCOM=NCOM+1
SUM=SUM+AVG(N)
SUMN=SUMN+DNEW(N)
10 CONTINUE
IF(NCOM.LT.NLAP) RETURN
BIAS=(SUM-SUMN)/FLOAT(NCOM)
C**** Update period of valid data, averages and weights
DO 20 N=NF1,NL1
IF(DNEW(N).GE.XBAD) GO TO 20
WTNEW=WT(N)+WT1
AVG(N)=(WT(N)*AVG(N)+WT1*(DNEW(N)+BIAS))/WTNEW
WT(N)=WTNEW
IWT(N)=IWT(N)+1
NSM=NSM+1
20 CONTINUE
50 CONTINUE
RETURN
END

This particular bit of code was of special interest, because it was errors discovered in this part that ultimately led to Steve McIntyre forcing NASA to finally publish the code, after many years of resistance. Steve had managed to reverse engineer how the code works from its output – a brilliant tour de force of detective work. It contains the adjustment for merging fragments of series from one location that have come via different routes into a single series. What it does is to look at the two bits during the overlap period, find the difference between the averages during this time, and then add this offset to one of them before splicing them. It’s supposed to compensate for systematic differences between different temperature monitoring stations.

However, it sometimes happens that you get data from the same station via different routes, which get merged by the same routine. If there are transcription errors or missing values in one portion, the averages during the overlap will be different, and one bit will be shifted relative to the other, even though they are exactly the same data. This introduces erroneous trends into the result.

The size of the error that resulted was not particularly significant, but the point of the discovery was to show how poorly the code had been validated. As you can see from the above snippet, the variable names are short and not meaningful, there are few comments and they’re very terse, there is no documentation, there is dead code from previous versions left in (The “50 CONTINUE” line isn’t called from anywhere and doesn’t do anything), there’s no error checking, the computer language (Fortran 77) is archaic, poorly structured, and hard to read, and it is littered with a spaghetti of GOTOs jumping everywhere. It’s hard to understand, even for a coder.

There are several other problems with it too, to do with integer division, but I won’t get into that now.

This standard of code would get spat back with a ‘fail’ in even a first year computer science homework. And yet this is what NASA use today to generate GISTEMP, one of the most prominent temperature series in the world.

Anyway, now that the code is out in the open, a bunch of software engineers are busy re-writing it to modern coding standards. See their presentation on progress here. I believe NASA have already said they’ll be adopting it as soon as possible.

Besides NASA, there are also a range of other temperature reconstructions from various groups, and yes, I have looked at them. Several of them use Mann’s output as one of their inputs – in particular, the noamer1 series that Mann constructed for North America, and which is dominated by a small group of Bristlecone pines in one small corner. (And which were already known not to be temperature-sensitive.) The reconstructions that don’t use Bristlecones almost all use another series called Yamal, which was recently discovered to depend for its hockeystick-shaped signal on a single tree. (Called YAD061 in the record.) The story of how the Yamal data finally got published more than ten years after it was first requested is also a fascinating story, but too long to do it justice here.

Seriously, if you’re interested in knowing the full story, there’s a book called The Hockey Stick Illusion that tells the entire tale. If you want to be able to argue that the sceptics are wrong, you need to know what they’re saying first.

This is flat out wrong. Peer reviewers absolutely do check calculations, check references and check observations. They may not have access to specific samples or equipment required to validate whole experiments. But they certainly do check for accuracy. Even if all of those checks work out, reviewers often insist that a paper include additional analyses or different methodology to back up conclusions. Reviewers are also chosen because they have enough experience in the field to see fundamental flaws in an experimental design.

And in cases where the data are public, codes are widely available and the study largely mathematical (like much of the work in climatology) it is possible to validate study results entirely.

It is a far more cursory examination. Is it going to be of interest to the journal’s audience? Is it novel and useful? Does it provide enough detailed information for a reader to be able to tell what was done, replicate, or check? Does the weight of evidence support the claims?

What you are describing is the job of the editor, not the peer reviewer.

I’ve done peer-review in the past, so I do know.

Well then you’ve done it wrong.

There’s a lot more supporting information out there. You say you have a scientific background. I suggest you do your own study, but try to avoid letting bias get in the way of finding out the truth. That’s the main problem with people like McIntyre. He is biased against AGW theory so all his work is muddied by it, and so it can not really be trusted.

NASA has a lot of climate change information and their work is probably the most bullet proof stuff there is. It might be a good idea to see how they used their own data and methods to arrive at the the same hockey stick as Dr. Mann. Several other have done this independent verification work too so they might also be worth taking a look at.

#108.
Thanks for the link. But I’m after something in a bit more depth.

Regarding this particular page, I noted the bit where they say ice ages take 5,000 years to warm 7C, so the recent rise is 8 times faster than previous warming. This is incorrect.

Temperature shows different rates of warming on different time scales, because it is a fractal process. It’s a bit like asking how long the coastline of an island is – look at it coarsely and you get one number, look at the finer detail and you get a much bigger number, measure round each rock and pebble, and you get a huge number. Temperature works the same way. The rate of rise or fall can be much faster over short intervals.

Here’s an easy experiment to do, that is well worth it for the intuition it gives you about such processes. Use a spreadsheet to generate a series in which the first term is zero, and then each term after that is 0.99 times the previous value, plus a zero-mean random number.
In Excel for example, you can do this with the formula
A1 = 0
A2 = 0.99*A1 + NORMSINV(RAND())
A3 = 0.99*A2 + NORMSINV(RAND())
A4 = 0.99*A3 + NORMSINV(RAND())
etc.

(or if you find it easier to follow, you can use A2 = 0.99*A1 + (RAND()-0.5).)

Continue it for a few thousand lines, and then plot a graph of the data.

You will see that over long periods things are fairly slow and flat, but as you look at shorter and shorter intervals, it gets wilder with long, steep rises and falls. The behaviour over short intervals can vary much more rapidly than it does over long ones.

(Note, I’m not saying that temperatures follow this particular sort of process. This is a simple example to build intuition about stochastic persistence. The models they fit to simulate real temperatures are a bit more complicated. Nor am I saying that climate is just random numbers – it’s just that the output of chaotic processes often act as if they were random.)

It’s also worth changing that 0.99 to a zero, so you get pure white noise randomness. Which graph looks more like temperature data?

Another thing worth knowing about is the Central England Temperature series – known as HadCET to the trade. This measured temperatures in England back to the 1600s, and it shows several extended trends. The biggest, from about 1680 to 1733 was bigger, longer, and steeper than the rise in temperature in the modern period. It started from a much lower point, and peaked just short of the present day maximum.
(You can download the data from the internet if you want to check.)

The problem of course is that while we know that on a local scale the modern warming is not unprecedented, we don’t have the worldwide records to be able to say whether 1733 was a global phenomenon.

#105.
This article is incorrect. Under the scientific method, when results are published, the rest of the scientific community examine them, attempt to replicate them, knock them down, extend them, and so on. If a published result subjected to this barrage survives the process, it gains in credibility. If it can’t be replicated, or alternative explanations are found, or the methods are shown to be flawed, then it is rejected. It stays in the literature, but scientists who know the literature also know that it is not to be relied upon. This extended process, that can take years or decades, could be described as ‘peer review’.

But journal peer review is entirely different, serving a different function. Back in the old days of science, scientists communicated their results by writing letters to one another, but because this was so inefficient, they got together into societies that acted as central distribution points. Letters and results were submitted, and all the members of the society could read the published journal. The journals, hungry for subscriptions, were keen that papers published would be interesting, relevant, and competently done for their audience, so the editor would select what he considered the best papers. Occasionally, if he didn’t have the relevant expertise, he would ask the opinion of an outside expert. It wasn’t until the 50s if I remember correctly that external peer-review was expanded to the system we have today. But a peer-reviewer does not check a paper for accuracy – he does not re-do calculations or experiments, analyse data, chase references, or check observations. It is a far more cursory examination. Is it going to be of interest to the journal’s audience? Is it novel and useful? Does it provide enough detailed information for a reader to be able to tell what was done, replicate, or check? Does the weight of evidence support the claims? (As in Sagan’s dictum: extraordinary claims require extraordinary evidence.) Does it contradict other work or standard results in the field, and if so is the conflict addressed? And does it appear to be competently done, with no obvious errors? Basically, it asks the question: is this worth your time looking at? It offers no guarantee that it is correct.

Journal peer-review is an editorial function. Rather than the entire community of scientists, it is conducted by a mere two or three, over a couple of weeks, of part-time unpaid work. Journals and peer-reviewers can often have their own professional prejudices. (If you work in academia, you get to know what these are and what journals will publish what.) And a paper can be rejected for not fitting the journal’s intended audience as easily as for any errors.

I’ve done peer-review in the past, so I do know.

So you see, when Steve McIntyre asked to download Dr Mann’s data, so that he could try to replicate it, that was the scientific process moving forward. And when he found problems with it, that ought to have triggered the community to have another look, and if necessary revise it or even throw it out. The fact that they haven’t shows that something has gone wrong with the scientific process in this area.

This is not really all that unusual. During the 1930s, the scientific establishment led by the British astronomer Eddington rejected black holes as a possibility, after Chandrasekhar first demonstrated them as an inevitable solution to Einstein’s field equations in general relativity. Other prominent physicists, such as Pauli and Bohr agreed with Chandrasekhar privately, but would not support him publicly against the astrophysical community. As Max Planck said, “Science advances one funeral at a time.” It took several decades before black holes were generally accepted. Chandrasekhar eventually won the Nobel prize for his work.

Scientists themselves are human. Science will self-correct eventually; it always does. But it may take some time.

NASA has a really nice Q & A page up. It might help to answer your questions.

http://earthobservatory.nasa.gov/blogs/climateqa/if-earth-has-warmed-and-cooled-throughout-history-what-makes-scientists-think-that-humans-are-causing-global-warming-now/?src=eoa-ann

Thank you. Yes. They say essentially that “A large number of scientists believe…”, which is honest, at least. They correctly state that the primary evidence relied upon is model results from GCMs, they correctly say that the reliability of these has been called into question, but then unfortunately they answer that with a “correlation implies causation” argument based on the modern rise in temperature.

They essentially agree that all my concerns and questions exist, are valid, and not fully answered, but go on with a “nevertheless…” and cite the IPCC’s opinion and the precautionary principle. (Which I’ve always considered to be a modern version of Pascal’s Wager.)

So I’d say it was a fair statement of the position, but it doesn’t actually answer my questions. That’s a lot better than many, though.

http://downloads.climatescience.gov/Literacy/Climate%20Literacy%20Booklet%20Low-Res.pdf