Jones and the Climategate data: when was it destroyed?
The blog Wattsupwiththat makes an excellent point (hat tip: commeter “huxley”)—one that the press might have pointed out as well, had its members been the least bit interested in doing their jobs: if the raw data was actually destroyed back in the 80s, why do all of Jones’ subsequent emails indicate it still existed, that he was loathe to release it, and that he would even prefer to destroy it rather than have it fall into enemy hands?
These are the possibilities: either it existed all those intervening years and he destroyed it more recently, or it was destroyed back in the 80s and he didn’t want to admit that fact and leave CRU open to criticism. In other words, either he was lying then, or he’s lying now.
[NOTE: And yes, I know that “data” is plural. But in common usage in the US, it’s usually treated as singular, which I’ve done here.]
While visiting Watts Up With That, don’t miss this post:
http://wattsupwiththat.com/2009/12/04/a-devastating-response-to-theres-nothing-to-see-here-move-along/#comments
This post quotes, in its entirety, an articulate and authoritative comment posted on a Climategate article in the Boston Globe by someone named Sean who describes himself as a climate scientist. Sean lays out the meaning and impact of Climategate in absolutely scorching terms. Whoever he is, his thoughts are not to be missed.
I second Mrs Whatsit on that WUWT post and WUWT in general. For those who are seriously interested in the AGW controversy, “Watts Up With That” a must-read.
My bet is that Jones & Co. destroyed the data recently.
Back in 1984 an IBM magnetic tape cartridge was 4″ x 5″ x 1″ and stored 200 MB.
According to Gavin Schmidt of RealClimate, 95% of the adjusted CRU data or its equivalent is available at http://data.giss.nasa.gov/gistemp/ .
I’ve been to that site and based on the various download pages, that data plus various software programs is 100 MB tops. Now maybe there was additional metadata and other source materials with the raw data but surely those wouldn’t have gone over 100 MB.
In other words the original data would easily have fit on one 200 MB cartridge. Throw in a couple of backups and the original data wouldn’t have taken up more room than a decent hardback dictionary.
I don’t believe the CRU scientists idly lost or threw out that data while housecleaning or moving to a new building.
What I have read from the CRU Hack emails and code, and seen of these characters over the years indicates that these AGW scientists are more dishonest than incompetent.
An anecdote from my past may shed a bit of light on how difficult it will be for the Warmists to give up their scenario of catastrophic AGW. The commenter, Sean, at WattsUpWithThat reminded me of this.
When I was a geology student 55 years ago the head of the department, Dr. T., had done his dissertation many years before on a well known sandstone formation. Dr. T. spent many years studying beaches and their depositional patterns as well as sand dunes and riverine sand deposits. His conclusion as defended in his thesis was that the formation had been laid down in massive, extensive beach deposits. His theory and dissertation went unchallenged for 40 years or more until another professor, Dr. C., obtained a new, high-powered microscope with which he could examine thin sections of the sandstone in great detail. Dr. C. concluded that the sandstone had been laid down as an ancient and extensive desert sand deposit because the sand grains were angular and pitted, unlike beach sands, which were more rounded from constant washing against one another. Dr. T. and Dr. C. had been great friends who fished and skiied together. However, once Dr. C. challlenged Dr. T.’s theory they quit speaking and never reconciled. Dr. T. continued to defend his theory, in spite of the new and, most believed, stronger evidence, right up to his death.
A mind can sometimes be a difficult thing to change. It will be the same with AGW.
A.J. Strata, whom I have found to be an honest and diligent blogger, is deep inside the CRU emails and code:
Proof CRU Did Not Lose Their Raw Station Data
Huxley: what is your take on this?
http://scienceblogs.com/deltoid/2009/12/quote_mining_code.php
Whatever happened to the data “publicly” I don’t believe it was deleted or destroyed in reality. No scientist in his right mind would do that, and I believe that Dr. Jones, despite the many professional behavioral problems he seems to have, is enough of a scientist that he couldn’t countenance that. I would lay money that somewhere – on a CD, a USB thumb drive or some other small, unobtrusive device hidden in some out of the way place – you’ll find all the “destroyed” data securely recorded. It just may never see the light of day again – or, if so, perhaps not for many years.
Neo: “had its members been the least bit interested in doing their jobs
That’s a point that’s become glaringly obvious in the last year. The journalistic mores of the press have slid so far that if they are uncomfortable with some story that conflicts with their beliefs or their desires, they simply refuse to cover it – their professional obligations be damned. Once may be a fluke, twice may be coincidence, but by the time it happens a third time it starts to form a pattern.
They simply ignored the Van Jones story because they didn’t want to report anything that would hurt the “O”. Remember the lame excuse given by, was it the AP?, where they said that because of the holiday weekend they didn’t have reporters available to cover it? It sounded like a lame excuse at the time and that conclusion was only borne out by future developments. When the ACORN video story broke they were completely AWOL again. Was it another holiday weekend? Or did they just not want to cover a story that made their side look bad?
Of course, they made some noise about making more of an effort to be attuned to things going on in the “right wing blogosphere” and all that, and the NY Times appointed a super-secret editor to watch what the other half of the country was talking about but now the exact same thing has happened again. I haven’t researched this personally, but from what I’ve read, the number of mentions of the Climate Emails story (see how I didn’t even have to call it Climategate) in the MSM is practically countable on one hand. Most of the main MSM outlets simply haven’t reported it. Many not even once. It’s breathtaking.
It apparently isn’t a very important story even though:
1) The IPCC head has called for an investigation
2) one of the leading IPCC authors has been suspended and is being investigated by his university
3) the hockey-stick man is being investigated by his also
4) there is clear evidence of malfeasance of various legal and scientific kinds in the emails
5) other climate scientists have come out criticizing their colleagues
6) all this is directly connected with one of the biggest and most far-reaching stories of modern times
7) which might turn out to be the most expensive thing mankind has ever undertaken
and not one of these MSM outlets thinks it warrants ongoing and probing coverage. Even though it’s clearly the topic of conversation for millions of their readers. Again, simply breathtaking.
I loved this line I heard the other day: “Journalists used to be people who told their readers the news. Now it’s the readers who are telling the journalists the news.”
No wonder the major media is tanking. Who would patronize any business where the employees gave such crappy service? “I don’t feel like doing my job, so I won’t.” isn’t a great operating principle for any enterprise wishing to be successful.
(Of course, the actual pattern of journalists ignoring news stories was really set in motion last year with Reverend Wright and all the incuriousness shown towards Obama’s past. Dereliction of duty is not a pretty thing to watch.)
Gringo: I’ve been following the discussions at ESR’s blog and also followed a link to that Deltoid page.
Yes, the code portion that cooks the data to produce a hockey stick is commented out, i.e. does nothing when the computer runs that code file.
However, the fact that the code portion exists and has not been deleted from the file means that it ran once upon a time and has been preserved in case it might run again.
Perhaps it was harmless scaffolding code the programmer used before he could write the complete correct code. Perhaps it was code that ran when they needed some quick hockey stick graphs before the “value-added” data was in place to produce the “right” result.
Unless we get Harry under oath, we won’t know. But it doesn’t give one confidence. At the least it is evidence that these boys knew they were looking for hockey sticks.
Speaking as a programmer, what bothered me most about the HARRY files, was first, “that’s-a-some-sloppy-code,” and then Harry’s comment:
“… yet another problem that’s based on the hopeless state of our databases…”
Help!
Academics are usually poor programmers and mostly it doesn’t matter much. But now we are being pressured into restructuring the world economy and adding substantial carbon taxes to all first world citizens.
I say we need to hit the RESET button, redo all the AGW work from the ground up with real programmers and skeptic scientists watching every frakkin’ step.
Hot damm: UK Met office announces a do-over: entire global temperature series — 160 years worth!
For those that still think Climategate has no significant impact on climate science, this revelation tells another story.
The Met Office plans to re-examine 160 years of temperature data after admitting that public confidence in the science on man-made global warming has been shattered by leaked e-mails.
The new analysis of the data will take three years, meaning that the Met Office will not be able to state with absolute confidence the extent of the warming trend until the end of 2012.
Furthermore:
Since the stolen e-mails were published, the chief executive of the Met Office has written to national meteorological offices in 188 countries asking their permission to release the raw data that they collected from their weather stations.
The Met Office is confident that its analysis will eventually be shown to be correct. However, it says it wants to create a new and fully open method of analysing temperature data.
This will end the cute game Gavin Schmidt kept playing by saying that some of the data was unavailable because it was proprietary. My point was that if global warming was such a terrible threat, ways can and must be found to get beyond such agreements.
Implications:
Oh, wow. I just had a thought about the EPA’s impending power to regulate CO2 (I really hate not being able to subscript the 2, there) as a pollutant. Humans exhale it with every breath. When this fact is coupled with some kind of pollutant control power, are we going to see the EPA empowered to control human reproduction? Offspring will destroy the earth–hence, they will not be permitted. Carol Browner would love it.
I know it sounds extreme and improbable. Still, we should not underestimate the dedication of the committed. . . .
It can’t happen here, right?
Sheesh.
Kudos, Huxley. You’ve been doing excellent research, here.
I’m having a hard time believing that these guys are going to be able to shut this thing down. It’s spreading like a virus, and ABC, CBS, and NBC are not the world.
Well, we can hope anyway that they are not. I’ve been telling friends at work that this is too big to simply drop from view. But the truth is, I’m not sure.
I’d rather think that truth will out. But I’m not prepared to bet anything I can’t afford to lose on it.
betsybounds: Strictly speaking, carbon dioxide produced when we breathe out or when plants are burned is part of the normal load of global carbon in play, hence does not contribute to global warming.
It’s when we liberate CO2 from long buried fossils that we increase the global carbon in the atmosphere and thereby contribute to global warming. To reduce the carbon in the atmosphere, we need to find a way to extract it then bury it in the ground, which is actually suggested by some.
So, as inviting as it is to suggest AGWers do their bit to reduce global warming by ceasing to respire, it doesn’t work as a smackdown.
I’m having a hard time believing that these guys are going to be able to shut this thing down.
betsybounds: Indeed. It’s too late. The governments, the MSM, and the AGW apologists have done their damnedest but the genie is out of the bottle.
The ordinary folk were already skeptical given the lack of warming in the past ten years and their innate expectation that the government is always looking for ways to extract taxes. But this finishes AGW unless they can rebuild that edifice openly and transparently.
My European friend tells me that global warming is commonly slagged in some of the most popular UK radio and television shows. Terry Wogan (“Sir” to you) is the most listened to media personality in Europe and he is always making sniggering comments about global warming.
Huxley,
I take your point. However, given that population control and carbon reduction are both parts of their program, it wouldn’t surprise me to see them try to synthesize a link between the two. Remember that, at this point, it’s more about politics than science.
I prefer, however, that you are right.
When McIntyre re-ran the statistical work that was used to construct the famous Hockey Stick he could not reproduce the blade unless he cherry picked the data. He only recently got the data, so this is fairly new. The data needs to be made available to anyone who wants to check it and see if they can reproduce the work of Mann, Briffa, etal.
Right now things are in an uproar as the “Deniers” gloat about the e-mails, code, etc. while the AGW faithful (with an emphasis on the FAITH) look desperately for ways to explain away what is, at the very least, very bad faith by Jones, Mann, Schmidt, Trenbarth, etal.
Like you, huxley, I want to see all the data and climate model codes put out there for other scientists to run and verify the results.
Dr. Roy Spencer’s take is, I think, a good one. T
He believes there is no basis for believing CO2 can cause runaway warming as trumpeted by the AGW faithful unless someone can prove that there is some component of the atmosphere that multiplies the effects of CO2. The AGW modelers have put “forcings” in their models to come up with that result. It is their idea that as the atmosphere heats up it becomes more humid because of greater evaporation. They posit that this is a positive forcing that will multiply the small effect of CO2. The problem with this is – precipitation in the form of rain or snow tends to release excess humidity and heat from the atmosphere. Accounting for this effect in models is difficult because it is not uniform nor well understood on a global basis. Also clouds have a huge effect on heating and cooling but are so variable in time, characteristics (size of droplets), and location they are difficult to model as well. Another wild card is the oceans. Circulation patterns, upwelling of cold water, the Pacific Decadal Oscillation and the el Nino, la Nina patterns are just a few of the things that influence climate and temperature that are difficult to account for in a model. The general idea that Dr. Spencer has is that oceans, precipitation, and air circulation all tend to work to cool the atmosphere and increased heat from the sun is the logical place to look for increasing temperatures. So, the case for AGW is a long way from being positively made. For a more copmplete discussion of this read Dr. Spencer’s book, “Climate Confusion.”
A google search for climate change denial will return over 70,000 hits. Environmentalists, journalists, and some climate scientists, convinced of the nobility of saving the planet from AGW, are prepared to demonize anyone who dares to disagree. The leaked e-mails, the lack of interest by the MSM, along with blogs like Real Climate certainly show this. It needs be made evident to the public that AGW is a theory that may or may not be true, and we should not destroy our economy until we know with more certainty if a climate disaster lies in the future. Not easy to do if the MSM doesn’t tell the story straight.
betsybounds: Going back to the fifties and sixties, it’s been clear how horrified environmentalists and their fans are by human population. Many are surprisingly honest in their wish for a big die-off.
Even in my leftie days I always felt a bit … squeamish … about that.
You are right on the money that the bottom line for many of them is the need to reduce human population. If one buys AGW as a big problem, reducing the number of Americans especially is about the most effective way to reduce global warming.
A big reason I don’t have a family is that I believed that stuff when I was younger.
there is no basis for believing CO2 can cause runaway warming as trumpeted by the AGW faithful unless someone can prove that there is some component of the atmosphere that multiplies the effects of CO2.
JJ: That’s their reasoning and there may be something to it … to some extent.
But nothing is simple with such a huge, complex system as Earth’s climate. If such a minor gas in the atmosphere could cause a runaway greenhouse effect we would have gone the way of Venus long ago. Our planet has seen much higher amounts of CO2, around twenty times as much, in the past without succumbing, so I see no reason for that to happen now.
Besides the real threat that goes unmentioned is the next Ice Age and that, rest assured, is coming as sure as April 15 and taxes, and it is close to a civilization killer. Imagine much of North America, most of Europe and Asia buried under a quarter mile of ice.
J.J.,
You are correct, of course. However, and additionally, these guys have advanced and blessed the notion that they call the Precautionary Principle. It allows them to modify scientific definitions to such an extent that all one need do is suggest potential harm, and that suggestion alone becomes sufficient cause for shutting down human activity. The Precautionary Principle has gained a fair bit of currency–my nephew, for example, who currently holds a tenure-track position at a private New England university, is among its devotees. It’s intuitively unexceptionable, but works like a parasite. The thing is, who could possibly object to making sure no harm is done? Its crippling essence is ignored.
Well anyway, we live in interesting times. I guess some would say that that means we bear a certain curse. 🙂
The Precautionary Principle has gained a fair bit of currency…
But only because a lot of these functional lightweights don’t know that you have to apply the Precautionary Principle – otherwise known as a risk-benefit analysis – to the “precautions” themselves, also, which makes it obvious that the alleged cure is almost certainly much worse than the alleged GW disease, a “disease” which itself has not even been shown to be a net diseae – because just about all the ipcc did in this regard was to disasterize GW, regardless of the cause and demonize CO2. Which are another couple of examples showing that Climate Science is not doing Science.
It might be true that a lot of these AGWers never think about what might go wrong with their “good intentioned” fetishes or avoidance actions in the face of their phobias, but this only means to me that they are not even good intentioned to begin with.
But only because a lot of these functional lightweights don’t know that you have to apply the Precautionary Principle – otherwise known as a risk-benefit analysis – to the “precautions” themselves…
J. Peden: Amen!
Even if the AGW scenarios outlined by the IPCC were true, we still need to have a debate about the cost/benefit/risk of each scenario and whether spending 1% – 5% of global yearly GDP to reduce carbon emissions is the best response.
But, no, AGW advocates want to skip all the steps of settling the science of AGW and discussing its remedies, and instead go straight to world government taxing and policing and wealth redistributing and do it now, now, now.
I prefer not to go all “They’re trying to steal my money and put me under the jackboot of the One World Environmental State!” But danged if their eagerness doesn’t look that way.
“…Many are surprisingly honest in their wish for a big die-off…”
But never for themselves, strangely enough. I continue to hope that some of these lefty “scientists” will decide to set the example and “take one for the team” but have so far been disappointed.
waltj,
its not like they dont have easy access to nice ways to do it… think of alan turing… gay, and went as snow white… he ate the poisoned apple. (its hard to be reverent when someone chooses to go that way)
huxley and J Peden:
Paragraph 2 of this Nature editorial is the perfect illustration of what you are discussing:
http://www.nature.com/nature/journal/v462/n7273/full/462545a.html
Stephen Hayes at Weekly Standard has done a good job on checking the E-mails. The article is definitely worth a read.
I should have added that there is nothing in any of the scientific data that say anything about the particular climate bill being discussed. It amazes me that lefties never look at thepossibilties for corruption in the legislation they propose–or do they?
“I don’t believe it was deleted or destroyed in reality. No scientist in his right mind would do that,”
And no Priest in his right mind would touch children.
The cultural stereotype of scientists has them losing stuff all the time. They had a couple of decades to misplace the original data.
A more recent loss bumps against the idea that these folks were in their “right mind”. They have been anointed and self-anointed as priests for gaia. Science is their secondary faith.
Or, it could be similar to Saddam’s WMD. Everyone believed the raw data existed, even though it had been lost/degraded/discarded years ago. Those with power wanted everyone to believe there was data, as belief was all that was necessary to extend their power. At some point the lie becomes habit and a personal “truth”.
Earlier Gringo asked me about a pro-AGW “bust” to Eric Raymond’s horror at some CRU code that cooks temperature data. PhysicistDave is a contributor to the coding topic but he doesn’t write much about code. He is literally a student of Richard Feynman and takes on two AGW advocates as a true scientist.
One can learn much just by searching through the topic for PhysicistDave’s posts. Here’s a sample:
The defenders of the GCMs [General Circulation Models — the models AGW scientists use to predict future climate –huxley] are basically saying, “If you critics think the GCMs have errors, point out those errors and improve the GCMs.”
On the other hand, a lot of us who are skeptics simply doubt that a successful GCM is possible. We are not willing to spend our time “improving” the GCMs, because we do not think there can be an adequate GCM, given the current limits of science and of present-day computers.
A few years ago, one of my close family members was seeing a fortune teller (and paying good money for the privilege); I was told that the fortune teller’s abilities were “amazing.”
I had a simple response: “Please ask the fortune teller to tell us who will win the World Series and the Superbowl for each of the next five years.”
I was, of course, told that the fortune teller did not make *that* sort of prediction: no, she only predicted fuzzy “personal” sorts of things that could be interpreted in various different ways as the listener chose.
Right.
At that point, I knew I did not need to investigate in detail the fortune teller’s past predictions, the scientific basis for her supposed powers, etc.
I knew she was a fraud.
That is the beauty of science: we do not need to argue about whether the Holy Spirit truly inspired our fellow scientists, whether they have shown saint-like powers, etc.
No, all we need is some new and quite unambiguous predictions that actually turn out to be right (or wrong), and we can thereby judge the quality of their science.
— PhysicistDave
huxley Says:
December 5th, 2009 at 9:40 am
That’s pretty much been my assumption for years now, ever since I first heard of AGW.
I’m not sure what motivated the scientists involved, whether they were seduced by the opened floodgates of government grant money or whether postmodernism has finally gained a foothold in the hard sciences along with the humanities. Both could be plausible explanations.
But the way political leaders the world over eagerly latched onto it and started shoveling money at the compliant scientists, along with the mostly uncritical coverage of the issue by the erstwhile ‘mainstream’ media, well, that looks a good deal worse than unseemly, in my opinion.
“Never let a crisis go to waste.” Even if sometimes you have to manufacture the crisis.
I’m still not 100% certain that we’re out of the woods yet. While it looks like Cap & Trade is DOA, and Copenhagen will probably end up being a statement of platitudes instead of the foundation for world government, I wouldn’t put it past the politicians to try to ram something through, even if it isn’t everything they wanted. The statists and globalists are so close to their goal that they can taste it. It must be awfully frustrating for them to be thwarted at the eleventh hour.
Still, I think our chances of avoiding that fate are significantly greater than they were two weeks ago, before Climategate erupted.
After reading the Nature Journal article, it struck me how confident they are that if the data and codes are all available for running and testing that they will all be verified as correct. If that actually occurs, I will reverse my opinion about the issue. That said, after years of studying the issue, even though I am neither climatologist nor mathematician, I am just as confident that independent study of the codes and data will not verify AGW as catastrophe.
The precautionary principle is, as you say, betsy, their safe harbor. It’s their ace in the hole that they believe trumps the conclusion that we just don’t know if CO2 can cause such outsize effects. However, J. Peden reminds that what is really needed is a cost-benefit analysis. That cost-benefit analysis has already been done by Bjorn Lumborg in his book, “Cool It.” His analysis shows that, even if global warming’s worst case(+7degrees C in the next 100 years – very few accept this figure) occurs, we are economically better off to adapt rather than try to mitigate. Though many of his ideas are socialistic, I agree with him and his reasoned arguments against crippling our economies in a costly, ineffective battle to drastically reduce CO2 emissions.
Still, I think our chances of avoiding that fate are significantly greater than they were two weeks ago, before Climategate erupted.
how so? they just ignore us and sign… nothing we can do then, unless we want to show obama isnt really president…
I hate to bring this up, but can someone explain how Charles Johnson can totally ridicule this issue? Can’t he read? Or is it that he is such a true believer in global warming that he cannot even allow an honest discussion of the so-called “Climategate”?
Is this how all those other true believers, the ones with real power, are just going to pretend that nothing is wrong with their pet belief?
I forgot to identify Charles Johnson as the boss of Little Green Footballs.
The MSM ignoring this story is almost as big as the bombshell story itself. My God…We have insanely corrupt people deciding humanities fate for the next century! Somebody pinch me.
Francesca Says:
December 5th, 2009 at 3:40 pm
No.
Yes.
Yes.
Hope that clears things up for you.
Artfldgr Says:
December 5th, 2009 at 3:24 pm
I did say in my comment that I’m not sure they won’t go ahead and do that.
I don’t know whether they’re prepared to take it that far or not. Because if they do, it will be proof positive that they are an illegitimate government, and we the people are fully justified in overthrowing it by force. Indeed, we would have a moral imperative to do just that.
Like I said, I don’t know whether they’re prepared to take it that far.
The ball is in their court. We’ll soon see.
Here’s a long, but interesting summary of Climategate:
http://weeklystandard.com/Content/Public/Articles/000/000/017/300ubchn.asp
I love this bit. Some are referring to the CRU e-mails as the “CRUtape” letters.
huxley and J Peden:
Paragraph 2 of this Nature editorial is the perfect illustration of what you are discussing:
http://www.nature.com/nature/journal/v462/n7273/full/462545a.html
Overall, that’s such a sad story indeed – Nature’s version brings tears to your eyes when you finally see the hassles and embattlement which the noble Climate Scientists have been subjected to as a result of their not doing any actual Science, no?
And Nature itself refused to make some of these these long suffering Authors archive their materials and methods in an accessable way, as per Nature’s very own requirements for publication, when it published certain Papers of interest to the evil [real] scientists and professional statisticians.
Which probably means that Nature itself did not “peer review” the very articles it published, or at least not so as to assure any further validity to the articles, while implying at the same time that these same articles thereby achieved the status of given truth, which, btw, never happens in Science and isn’t expected anyway.
Because the real “peer review” starts only after publication, as further proven by McIntyre, enc., when he was finally able to pry out the material and methods behind Mann’s Hockey Stick contention, or just got lucky when the Royal Society publication forced Briffa to archive his data in a recent paper – 10 years after Briffa’s first use of the data in a publication; which then showed “the decline” in tree-ring data from about 1940 on, which was later “hidden” by some “tricks” in ipcc and other official graphs, mainly for the purpose of trying to erase the Medieval Warm Period.
Nature has been a major part of the problem. It should never be read again, boo-hoo.
“Two weeks after the scandal broke, NBC Nightly News on Friday night became the first broadcast network morning or evening news program to inform viewers about “ClimateGate,” but only in the most cursory manner….”
http://newsbusters.org/blogs/brent-baker/2009/12/04/nbc-nightly-news-takes-climategate-frets-it-could-delay-taking-action
They are trying to change the free market
http://www.spiegel.de/international/zeitgeist/0,1518,665182,00.html