Two out of three Australians live in our capital cities where the longest and best resourced temperature records would be found. These are the places where the weather reports matter to the most people on a daily basis — and where headlines about records and trends will be widely discussed.  But these are also the sites which have been affected by the growth of concrete and skyscrapers, and potentially have the largest urban heat island (UHI) effect, so might need the largest adjustments.

Bob Fernley-Jones has been going through the BOM records for six of Australia’s state capitals, looking at the original raw data (at least, as is recorded in the BOM’s climate data online, called CDO). Bob compares the new “corrected” dataset called ACORN for these locations — that’s the all new marvelous adjusted data. He finds many step changes that can’t be explained by known site moves or the UHI effect. Many step changes occur in either minima or maxima, but not in both at the same time, which is also odd. As we already know, the adjustments usually cool the past — especially the minima (see all the blue lines on graphs below that dip below zero) — which has the effect of increasing the warming trend of mean temperatures.

For some reason thermometers that read 4 degrees C on crisp mornings in Perth circa 1920 should have read 2C  (or  4F lower), which was only discovered decades later. Maybe there is a good reason for that, but despite the BOM’s keen interest in saving the Australian climate, they don’t explain why these kinds of large changes are necessary in physical terms or with historic documents (indeed they don’t even seem very interested in the oldest historic temperature records we have). Apparently the mysterious process of “homogenization” with other nearby stations (which may be hundreds of kilometers away) is enough. We skeptics think that thermometers are not brain surgery, and that the BOM ought to be able to explain these large changes in terms of site moves, changes, or documented events. And of course it should explain because it is a public organization.

The BOM acknowledges that these capital city sites are affected by the massive growth of concrete and cars, and say these sites are “not used in regional and national analyses”. However, these sites are potentially used to homogenize other sites. (Which raises the question: do those inexplicable historic cooling adjustments then infect other station records, which are used in state and national trends?) The capital city records are also the sources of a lot of news headlines (like the “hottest ever night”). The media stories never mention how much the headline depends on adjusting the original records.

We’ve previously discussed the mysteries of the Melbourne temperature record but spent little time detailing Perth, which has the largest adjustments of all the six capitals studied. Thanks to Bob Fernley-Jones for his dedication. Don’t miss his summary paragraphs.

Since 30-40% of the Australian warming trend is due to inexplicable adjustments, I maintain my position that the BOM needs to be independently audited (not by a group selected by them), and their processing replicated (from raw data to final product) with all reasons and explanations made public, which is clearly not happening.  —  Jo

 

Corrupted Australian Surface Temperature Records (Part 1)

 A tale of Six Cities       Guest post    Bob Fernley-Jones
The term “corrupted” is used in the technical sense: has the data become so damaged it has lost validity?

a) In choosing the six capital cities with long records, (and so excluding Brisbane, and Canberra), there was an expectation that these sites would have the best resources, and probably the most robust records out of the 112 ACORN stations.   However, Adelaide, Darwin and Perth are effectively six separate sites because of substantial relocations from within town centres.  (They need consideration WRT potentially different environments including; wind directional exposure, cold air entrapment, and UHI effect.    There was little or no site adjustment in Hobart, Melbourne and Sydney.  The Melbourne (RO) site was appallingly bad for UHI etcetera and was finally closed in Jan/2015.   Sydney and Hobart sites are relatively open on grass, and it is specifically described in ACORN that there was no UHI effect in Sydney after 1910 and only a minor (vague) site adjustment in 1917.  There is no apparent justification for six subsequent step-changes.  See link below for site details).

b) There are many sharp step-changes that have no correlations in the ACORN Station History Catalogue.  There are also site changes that might arguably show step-changes but do not.  The mostly unaccountable random distributions are more chaotic in that they are different between minima and maxima.  However, there are tenuous arguments (e.g. Trewin et al) that a particular step change may be considered to only apply to either maxima or minima but not both, or, cause opposite effects between maxima and minima.  Whatever, it is unlikely that this is frequently validated.  There are also contradictions.  For instance a minor relocation at the Sydney Observatory in 1917 was claimed to reduce exposure that affected both maxima and minima, but a step-change is only seen in the maxima.

Summary points c, d, e and f continue below.

Notes on the graphs: These represent the size of the adjustments and are calculated as ACORN (new) – CDO (original). The blue lines are adjustments to minima, and the red lines adjustments to maxima. Lines running to the top or bottom just show points where data is missing. For all the details — see more expansive notes below.

Fig 1)  Adelaide…. Comments:  

Within the (blue) minima there are four step-changes prior to 1977, but they require careful inspection to see because of over-plotting by the (red) maxima.  But, during that same period the maxima only have two convincing step-changes versus four.  The ACORN site history gives three vaguely described site changes during that period but how the BoM could correctly determine their effect is unknown.  From 1977 when Kent Town replaced the original West Terrace site, there are three step-changes in the maxima towards ~2002, but no recorded site changes during that time.  Why the minima CDO are identical to ACORN back to 1977 but the maxima commonality only goes back to ~2002 is at least strange, (but by no means the most extreme case).  At Kent Town one of the step-changes embodies a notably different shape to the rest of the record.  That is to say, the “corrections” apply at different seasonal times.  Strange!
 

Fig 2)  Darwin…. Comments:

CDO is the same as ACORN way back to 1942 in the maxima, (after Japanese bombs permanently destroyed the original Post Office site), while the minima commonality goes back to ~1975.  (!)  There are three step-changes in the maxima versus none in the minima during those early ~32 years.  They have high displacements and atypical annual cycle shape.  Tree growth [progressive] at the Post Office is described in the ACORN site catalogue and those step-changes have no explanation.  Airport anomalies run 1942 to 1975.      Why is it that the equable tropical climate of Darwin has ACORN “corrections” over a range similar to that in Melbourne which is famous for far greater extremes even in one day.

 

Fig 3)  Hobart…. Comments:

Despite the site description below, Hobart station seems to be a relatively exposed elevated site at 51 m (167 feet) altitude.  It is arguably least effected by Urban Heat Island (UHI), within the periods of known anomalies.    It is irrational that for some 88 years prior to 2006 the average annual maxima have been raised uniformly by about 0.6 ⁰C/1.1⁰F, (by eye).  And, similarly in the minima going back from ~1966.  Again, there is great difference in duration of CDO common with ACORN!    Why should summer minima extremes be corrected to be almost 2⁰C (3 ½ ⁰F) warmer for the first ~42 years despite it being an open site?  (photo with recent building above).

 

Fig 4)  Melbourne…. Comments:

The sharp step-rise in the minima between 1929 and 1963 is not explained in the station record and neither are the otherwise relatively uniform “corrections” prior to 1995.  Since this has been arguably a progressively warming city (UHI) site there should be no sharp and prolonged step-changes prior to the 1990’s except for any substantial sealed road widening at some stage, which might explain the 1929 step-up.  However, the maxima are unaffected, and the step-down in the minima in 1963 together with a smoothing of extremes thereafter is unaccountable.  It is irrational that past records are “corrected” upwards because UHI effect is the opposite.

The BoM admits to a discernible rise in minima with construction of tall buildings nearby in 1996/7 but they claim the “record summer highs” during that period were not influenced.  It is a little complicated in that those buildings also changed wind patterns. (E.g. the anemometer data were discontinued, replaced by that at the airport).   However, it is tenuous to suggest that those wind changes only affected minima, and not maxima.  Melbourne is known for heat with northerly winds, and cold with southerlies.  Those tall buildings would seem to hinder both northerlies and southerlies.  Additionally, they have, quite apart from their thermal mass, arguably high albedo resulting in middle day reflection onto the site and its closely surrounding paving etcetera.

Unfortunately, the anomaly record is bare during that complicated period. (CDO same as ACORN).

Fig 5)  Perth…. Comments:

The many differing step-changes in minima and maxima have no correlations with the ACORN station history, (not that the history is easy to follow, see link below).  In testing three ambiguous possibilities in change-over dates in the long overlap from the Regional Office to the Airport, only one at 1963 did not result in obviously corrupted data.  CDO and ACORN are common from 2014 back to adoption of an AWS in ~1996.

Perth has by far the greatest “corrections” prior to 1964 with an overall range of about 7⁰C (12 ½ ⁰F) by eye, and an annual average range of about 2 ½ ⁰C (4 ½ ⁰F).  Compare Melbourne above!

There is an area of interest highlighted which in expanded view reveals monthly step-changes in the daily data and several concerns.  Careful study shows varying monthly step-changes throughout the Perth record.

 

Fig 6)  Sydney…. Comments:

Between 1910 and ~1963 there are no step changes in the minima but four in the maxima.  Prior to 1917 the site was in a “more exposed” position that is claimed had originally ‘lower maximum and higher minimum temperatures’.  However, that is only evidenced with the maxima step-change at 1917, not in the minima.  Surprisingly, it is also claimed that ‘any urban influence on the data was already fully developed by the time ACORN-SAT begins in 1910’.  Yet, despite no other site changes between 1917 and 1982, there are five step-changes in the maxima.

Summary — more mysterious adjustments:

The main points a and b are summarized at the top, but there are many more questions raised:

c) It has been reported that ACORN “corrections” towards 1910 in the longer term rural records result in ever increasing cooling.  The opposite is mostly the case in five of the six capital cities, where ACORN increases the temperatures towards 1910!  However, to correct for UHI in the context of “carbon” = “global warming”, the temperatures at 1910 should be the baseline, and those towards 2014 should be REDUCED.  (In determining any REGIONAL TRENDS in “global warming”).

d) To elaborate more on c),

• This note on the CDO Site Network Map (previously the HQ network) relates: Urban sites have some urban influence during part or all of their record, hence are excluded from the annual temperature analyses 
• Ken Stewart advises:  Classified urban sites are not used in regional and national analyses.  (See BoM/CSIRO ‘Centre for Australian Weather and Climate Research’ reports CTR049 and 050).
• However Ken also says:  Melbourne RO is used as a comparison neighbor station for surrounding Acorn sites.  The same happens elsewhere- Mackay is adjusted using Townsville and for example Rockhampton and Snowtown use Adelaide.

e) It is questionable as to why in the maxima for Darwin, ACORN data are the same as CDO way back to 1942, (1975 in the minima).  Was “original CDO” changed to match ACORN, lost, or………?  In comparison, Hobart commonality in the maxima only goes back to 2007, (1966 in the minima)!  Why do they do it with such erratic preferences?

f) As discussed in ‘5) Perth’, it seems that the BoM develops a monthly step-algorithm for each step-change, with which the daily data are merged.  Oddly, here are two stations where they seem to have “forgotten” to merge the daily data.  I wonder what the algorithm might be that develops the monthly algorithms, given the illustrated wildly random shapes.

 

Methods, and notes on reading the graphs

This is a summary of part of a study involving some 50 Mb of data that was prompted by various controversies over the Australian Bureau of Meteorology’s (BoM) ‘homogenization’ of temperature records. The Bureau have made ‘corrections’ for nominal changes in site conditions over which criticisms have included that it has resulted in exaggeration of the reported warming trend, such as by excluding hotter data from before 1910; the starting point of their homogenisation.

However this is not about the methodology that the BoM has used in homogenization.  Instead it is a test for reasonableness in the resultant DATA after changes they made to their currently available “raw data”.  By raw is meant that recently retained under the BoM source; ‘Climate Data Online’ (CDO).

Still further controversy surrounds data earlier than the “raw data” used here.  However, it has been established alongside this summary that with all 24 locations researched so far having long records, that the homogenised data are undoubtedly based on the CDO “raw data” in those long records.  That said, many stations have short records and unfortunately much data in the shorter term have been made common between CDO and the homogenized ACORN- SAT (ACORN) files.

Nevertheless, any discovery of substantive corruption in that data is enough to say that the homogenization is unacceptable, regardless of what their methodology was or the “rawness” of the data used.

It is not possible to suggest what the BoM methodology should result in, because their processes have not been released to the public in sufficient detail.  However, it ought to be possible to validate if that data meets the required standards of reasonableness without knowing HOW they got there. 

In this part of the study, Six State and Northern territory capital cities are reviewed.

NOTES on the graphs:

The charts above visually illustrate anomalies in the BoM temperature records, by comparing downloads from two of their several data portals, namely; CDO and the homogenized ACORN files.

CDO daily data were subtracted from the ACORN version via digital processes and plotted in EXCEL 2010 spreadsheet software for 24 cities and remote rural sites, out of which six important sites are used here.  The anomalies are of great variety in their magnitude, shape and displacement.  (By displacement is meant the distance up or down of their centroids relative to the zero horizontal axis).  To assess comparative displacements, the vertical scale range is common in Figs 1 through 6.

These findings identify major problems with the BoM’s data processing methodology or skills (at least).

NOTES on reading the charts:

• The graphs are unusual in format.  Please do not skip these notes.  The red plot lines are daily maxima and the blue are daily minima on a highly compressed scale spread over up to ~38,000 days.  Some blue data may be smothered by the red which is plotted secondly.  In all cases an annual cycle is clearly evident because of seasonality.
• Those values that go up or down (in ⁰C) to the boundaries of the chart are the result of no data in either ACORN or CDO and thus are corrupted, (not anomalies in the sense as used here).  They are only of interest in indicating the completeness of the record.
• Note that the horizontal zero axis origin is not at the base of the chart but towards the centre because of both positive and negative temperature anomalies.
• In all cases, running back from 2014, ACORN is the same as CDO but for unexplained greatly varying periods.  Thus there are no anomalies (differences) in those periods, and only a red or blue line is seen atop the grey zero axis line.
• Negative temperature anomalies (ACORN – CDO) that are biased towards 1910 mean that the homogenization has resulted in an increased warming trend. (Vice versa for positive anomalies)

BTW, the regimented cycling seen in the following plots is a partial validation of the anomaly methodology, which is further established elsewhere (Supplementary information).

Conclusions:

The great variety of inexplicable problems exposed in the anomaly plots means that the data do not pass the test for reasonableness.   It is currently not possible to analyse why this is so, but whatever, the data have gravely inadequate credibility.

References:

ACORN Station Catalogue. (Including history of sites involved…. Sometimes two entirely different locations)

Sortable list of ACORN-SAT stations with linked data

Climate Data Online (CDO starting page)

Acknowledgements:

Thank you especially to Joanne Nova and Anthony Cox, Chris Gillham, Phill, Warwick Hughes, Bill Johnston, Lance Pidgeon, Geoffrey Sherrington, Ken Stewart

 Disclosures:

I’m a retired mechanical engineer with no past or present funding for my subject research from anyone. (Or interests other than seeking proper scientific methodology.)