When independent auditors found errors, gaps and deep questions about the HQ (High Quality) dataset for the official record of Australian temperatures, the BOM responded by producing a completely new set called ACORN in March 2012. But this set is also plagued with errors. One of the independent auditors, Ed Thurstan writes to me to explain that though the BOM says it aimed for the “best possible data set” and specified that they check internal consistency of data (one such check is to make sure that the maximum on any given day is larger than the minimum) when Thurstan double checked ACORN he found nearly 1000 instances where the max temperatures were lower than the minimums recorded the same day.
This raises serious questions about the quality control of the Australian data that are so serious, Thurstan asks whether the whole set should be withdrawn.
Why are basic checks like these left to unpaid volunteers, while Australian citizens pay $10 billion a year to reduce a warming trend recorded in a data set so poor that it’s not possible to draw any conclusions about the real current trend we are supposedly so concerned about. — Jo
The BOM goes to great lengths to assure us it’s high quality, peer reviewed, and rigorously checked, but with a days work, independent audits find major flaws
Anomalies Errors in ACORN_SAT Data
Ed Thurstan
July 14, 2012
Ever since the documentation for ACORN-SAT was released, I have had doubts about the ability of the Australian Bureau of Meteorology to honour their published intention to release all software that generated the ACORN-SAT data. ( I might amplify that thought later.)
In March 2012 the BOM released the report
“Techniques involved in developing the Australian Climate Observations Reference Network – Surface Air Temperature (ACORN-SAT) dataset CAWCR Technical Report No. 049 Blair Trewin
This specifies in great detail both the background to the development of the database, and the checks applied to the data. As Blair Trewin writes in the Abstract of this report:
“The purpose of this data set is to provide the best possible data set to underlie analyses of variability and change of temperature in Australia, including both analyses of annual and seasonal mean temperatures, and of extremes of temperature and other information derived from daily temperatures.”
I decided to take that document as a Program Specification, and write code to perform those data checks.
The very first check specified in section 6.1 of the above report is
“1. Internal consistency of daily maximum and minimum temperature
Since the temperature recorded at the time of observation (09:00 under current practice) is an upper bound for minimum temperature on both the day of observation and the following day (i.e. Tnd ≤ T0900,d and Tnd+1 ≤ T0900,d), and a lower bound for maximum temperature on both the day of observation and the preceding day (i.e. Txd ≥ T0900,d and Txd-1 ≥ T0900,d), daily maximum and minimum temperatures must satisfy the relationships:
Txd ≥ Tnd
Txd ≥ Tnd+1
If one or both of these relationships was violated, both maximum and minimum temperatures were flagged as suspect unless there was strong evidence that any error was confined to one of the two observations.”
In testing my code for the first of the two conditions specified above (which says simply that the maximum temperature recorded on any day must be greater than the minimum temperature recorded for that day), I found violations of this condition in the BOM data.
The following are extracts from the full violation log. The errors occur in many different sites and are spread across many decades:
In total, the ACORN-SAT database released in March displays about 1,000 (one thousand) violations of that simple rule that for any day
The Maximum Temperature must be greater than the Minimum Temperature.
This is a blindingly obvious type of error which should not have escaped quality control. It throws serious doubt on the whole ACORN-SAT project. In my opinion, these violations indicate that the entire ACORN-SAT database is suspect, and should be withdrawn for further testing.
Ed Thurstan
thurstan AT bigpond.net.au
July 16, 2012