CRU hacked emails

The recent news about stolen emails from the UKs climate research facility is a worrying development, not from the point of view of the science, but because of the political motivations behind the attack.
Many skeptics and deniers are using the small amounts of information gleaned from a tiny number of emails to back their political cause. I have had a look at some of the information, thankfully the contrarians do the filtering for us and quite happily pick out their favourite emails to discuss in forums and blogs.

One of my favourite snippets (which may or may not be original) is this one:

Added: email text removed for legal/copyright reasons. But the email basically suggested that some temperature data might have been removed from the end of a series for a graph to be used in a presentation, in order to hide alleged cooling. The text of the email is quite easy to find via an internet search.

This allegedly presents (to uneducated eyes) manipulation of data and scientific results. One could easily interpret the statement as a cover up or a conspiracy. Indeed, Nigel Lawson on Radio 4 this morning was doing the usual political spin (despite being in the wilderness these days) by denying he was a 'denier' and proposed that such comments in the emails should be investigated.

OK lets analyse this one troublesome email snippet (a few bytes of data, out of many millions that were stolen!). Firstly the person in question was talking about a presentation to other people. This means the graph mentioned in the email would be under scrutiny in public, so any points that would be missing would be noticed by any scientist or intelligent person in the audience. Not only that, but it is quite possible that the scientist explained why he removed the points during the presentation.

Now lets look at why a scientist would remove points when creating a graph. The issue is quite simple. Basically many curve plotting techniques can actually produce poor results if the data points at the end or the start of the series are radically different to the main batch of points in the middle. Indeed, the fact that the data suddenly stops in time, because we don't have temperature data for the future, there is obviously an artificial drop off in temperature even without specific data at either end of the sequence.

Filter techniques are sometimes used to produce a graph, this can result in the curve at the start and end of a plot to incorrectly drop off or even increase. With such plotting techniques it can even make sense to chop the graph off so that only the middle section is used. This has nothing to do with manipulation, what does happen however is that skilled scientists that know the characteristics of the maths used to plot a graph, can use that knowledge to choose the right methods that actually produce 'true' or accurate results.

So does a discussion about removing data really create a big concern?
In this context, it means very little and climate change deniers are being 'alarmist', basically using politics to manipulate the scientists and global policy. Unfortunately the scientists are being attacked vigorously by many people with some very dubious political motives. This is really a desperate act, partly for attention and partly because those opposed to the changes we need to make in our lives are getting to a point where anything could be justified, including poor science and criminal acts.

For more information about curve fitting try: Open Mind: Dangerous Curves