Cambridge University Press

The Australian Bureau of Meteorology may not be meeting WMO, UK, US standards

Since the Australian BOM allows for one second “records”, it’s not clear it is even meeting guidelines recommended for amateurs.

The key question: How much of the warming trend in Australia is due to the switch in the mid 1990s from older slower thermometers to new electronic zippy ones that could record every waft of hot air? How many records today are just noise?

If the BOM would release its calibration and comparison data instead of deleting it, we might know. Why won’t they?

Here’s an example graph from Maryborough where the daily maximum was 1.5C above every thirty minute reading. Ouch — are we writing outliers and noise into our history books and climate data bases?

Add “sampling method”  and averaging to your skeptical vocabulary. There will be a lot more discussion on these.

Maryborough. Graph by Ken Stewart.

Let’s consider some basic standards in the meteorology world

The Weather Observer’s Handbook 2012 tells us the new electronic sensors are more sensitive than the old mercury thermometers. The author, Stephen Burt, explains that the new electronic sensors can be too sensitive, and will record  “minor random temperature fluctuations“. This means they will simply “generate slightly higher maximums and slightly lower minimums” than the older equipment did. Temperatures from new style thermometers must be averaged over longer periods to make them comparable with the older mercury thermometers, and also to “iron out minor stray electrical noise or sensor logger resolution artefacts.”   h/t Chris Gillham.

The WMO, the UK Met office and the US all use longer average samples than Australia

The WMO recommend a one minute mean temperature standard be used, which is what the UK uses. In the US though, get this, meteorologists use a five minute average, and further below, Lin and Hubbard suggest it should be 7 minutes not five.

Further complexifying things — I hear the UK may put electronic sensors outside the Stevenson screens, which would make a big difference. Keep that in mind. UPDATED: Now I’m hearing, not so, it was just one photo.

Records, trends, daily ranges — they can all change with the sampling method

Was that really a record? Here’s a case where longer sampling times meant a record was not set on one day in Dodge city:

Weather Observers Handbook. Click to enlarge.

A good example is accorded by the maximum temperature recorded at Dodge City, Kansas during the heatwave which affected the southern and eastern states of America in summer 2011. Dodge City has one of the longest continuous temperature records in the United States, commencing in 1875. The hottest day on its long record stood at 110F (43.3C). On 26 June 2011 the highest 1 minute temperature observed was 111F (43.9C). However the value logged on an ASOS system was not accepted as a new record because ASOS take the maximum temperature as the highest 5 minute running mean, which was 110F. Thus the official high by the US method was 110F, tying rather than exceeding the previous record: by the WMO recommended method the maximum was 111F which would have set a new record.”

Here in Australia, with one second averaging, based on past behaviour, the new “record” could have been even higher, like 113F (45C) — who knows — which might have been measured for just one whole second. That kind of difference, which appears to be momentarily far above most of the data around it, has been entered into our official climate data online.

The Australian Bureau of Meteorology (BOM) responds with “Fast Facts”

In reply, the Bureau of Meteorology just published FAST FACTS: How does the Bureau measure temperature?  Jennifer Marohasy has already responded on her blog that the Bureau are rewriting the rules (read it all).

Lets compare the Australian method to the US. In the US, researchers publish long papers on, say, the bias that occurs with two different ways of doing 5-minute averaging (more on that extraordinary detail below). The WMO only recommends a one minute average. Prepare yourself for the Australian BOM’s answer about why they don’t even do that (get ready to be surprised). Engineers everywhere may be crying…

The guide recommends that temperatures be integrated over time to smooth out rapid fluctuations. There is more than one method of achieving this. The WMO guidelines do not prescribe which method to take. In its automatic weather stations the Bureau achieves this by using platinum resistance thermometers. These are comparable to mercury in glass thermometers.

Say what? “Comparable” to mercury in glass? That last sentence goes against everything in the WMO guidelines and the handbook. It blithely writes off peer reviewed papers, not to mention engineering-common-sense. Apparently the BOM are claiming a bucket defacto-averaging kinda-in-the-system. Because their electronic sensors are inside the Stevenson screen, there is a sort of averaging — a lag, or a low pass filter — but probably one that changes minute-by-minute with the wind-speed. They also claim that the response-time of the sensor is as long or longer than the changes in the temperature of the air. (Color me unconvinced, where is that data?)

As Jen Marohasy points out, this is far from a declaration that the electronic sensors are averaging temperatures in the same way that the old slow liquid-in-glass ones do. (Did I mention the data….?)

The BOM:

This means that each one second temperature value is not an instantaneous measurement of the air temperature but an average of the previous 40 to 80 seconds. This process is comparable to the observation process of an observer using a “mercury-in-glass” thermometer.

Spot the weasel-word “comparable” (in both paragraphs). Comparable is not equivalent. One and ten are comparable, but they are not equivalent. (Just like one second and sixty seconds, Hmm?) When Ken Stewart looked in detail at the Australian data he found a lot of volatility — in the most extreme case, temperatures fell by almost 3 degrees C in under one minute. In 44 of 100 cases, temperatures changed by more than 0.3C in the following minute. That doesn’t smell like “smoothing” or averaging. Think about what kind of overarching volatility it would take to create a smoothed average that still had that kind of instability.

The bottom line: What we have is 20 years of incomplete raw data that includes random noise recorded from instruments that are nothing like the ones used for about eight of the last ten decades in Australian climate history. Somehow we are supposed to connect that data together?

The Bureau has a budget of $365 million a year, yet it’s looking like they have thrown away the raw detailed data for the last twenty years that might have been useful. What we are left with is — we have to ask — possibly not salvageable?

Where to from here?

The first thing the BOM needs to do is store all the b—– data. Second, it needs to make that public. Then we need to talk about changing the BOM sampling process and getting some averaging done properly. Then we need to set up a search and rescue team to go back through the last twenty years of electronic data and see what they can do. At this point, it would take a group of PhDs years to set up replica stations in many different conditions to gather enough data to be able to kinda “back predict” what the electronic data should have shown had it used a proper averaging method.

A one second wafting hot spike,
Gives a false average temperature hike,
Which for sensors is crude,
And with past records skewed,
Are the methods the B.O.M. like.

–Ruairi

________________

The Nitty Gritty: How temperatures are supposed to be measured

Now we get into some more detail about how different the standards are overseas. We get a better idea of things that muck up the data, so we can all start to appreciate how tricky this is — and perhaps what better management and processes might look like.

Even the US five minute averages are not necessarily good enough

A paper by Lin and Hubbard discusses in extraordinary detail how a systematic bias occur. They have complex graphs showing how different sensors respond to changes in air temperatures at different temperatures, and with different wind speeds.  (TX, means maximum temp. TN means minimum.)

Liquid in glass thermometers are fundamentally different from electronic (platinum resistance) thermometers. They are not equivalent:

Although the LIG thermometers in the CRS also is an instantaneous observation for TX and TN the relatively large LIG thermometer’s time constant plays an exclusive role in filtering out high frequency temperatures and exponentially weighted smoothing temperature signals.

The bias (error) can be as large as 0.2°C (and they are talking about 5 min averages):

For the ASOS observations, the ASOS does have a 5- min running average for daily TX and TN, however, the even smaller sensor’s time constant in the ASOS makes its biases systematically significant (0.05–0.2 °C warmer on average TX or cooler on average TN).

Hence they recommend 7-minute averages:

.. “Commonly-used 5 min average was not sufficient for the fast-response thermometers in surface climate networks while the WMO standard thermometer (20 s time constant) should have a 7-min running average for reporting daily maximum and minimum temperatures. The surface temperature sensors with smaller time constant than the standard LIG thermometers must implement a follow-up running average algorithm.”

 Calibration alone is not enough to remove this effect:

It should be noted that most of surface temperature calibrations, even highly accurate calibrations, are unable to remove out this type of out-of-phase uncertainties in the TX and TN because the instrument calibrations for climate networks usually are of static calibrations rather than a dynamic calibration process for the electronic surface temperature sensors and the LIG maximum and minimum thermometers…

Is that warm bias bigger than the cool bias?

From the same paper: In the field the maximum temperatures (fig 8) are affected more than the minima (figure 9). Note that they are comparing two different ways to average 5 minute readings, and they still find a difference. They are not looking at the change from one-second to five minutes. That is something entirely different.

Daytime air turbulence intensity usually is stronger than that during nighttime, thus, it is understandable that the TX differences shown in Figure 8 were larger than the TN differences in Figure 9.

Maximum temperature bias from different sampling rates. Lin, Hubbard. Figure 8.

…

Minimum temperature bias from different sampling rates. Lin, Hubbard. Figure 9.

Thermometers circa 1912

The whole point of averaging is so we can compare modern temperatures with ones recorded a hundred years ago. So check out  Thermometric Lag (1912) which looks at the response time in different media including air at different wind speeds. The chart comes from a study published by the American Meteorological Society. Basically, if the wind is very still (slow in cm/sec) it takes a lot longer for a thermometer to respond to a change in temperature outside the screen. A thermometer in air may take over 3 minutes to respond on a still day. When wind speed is very fast, the change is more rapid.

..

Keep reading  →

The BOM’s bad luck never seems to end. Of all the 695 stations in Australia, 693 worked perfectly, but Jen Marohasy and Lance Pidgeon happened to live near, or have a personal random connection to the only two stations that didn’t — Thredbo and Goulburn. Apparently these stations had been flawed (not fit for purpose) for 10 years and 14 years, but the BOM world-class experts hadn’t noticed. I expect they were just about to discover the flaws when (how inconsiderately) Lance and Jen announced the errors to the world and the BOM were forced to do this pointless 77 page report to stop people asking questions they couldn’t answer.

The nub of this fracas is that something called an MSI1 hardware card was installed in cold locations even though it would never report a temperature below minus 10.4C. Awkwardly this doesn’t explain why the 10.4C appeared in the live feed, then was automatically changed to -10C in the long term data sets which are used for climate analysis. Does the BOM think the dumb public don’t know the difference between -10 and -10.4? Implicitly — the BOM installed the wrong type of card, and also accidentally had an error flagging system on top of that, that compounded the error by ruling out even the already-flawed -10.4, which may have been even colder. A double flaw, and both non-randomly warming the minima. What are the odds?

And John Frydenberg, Minister of Critters, Plants and Green-stuff  believes this? Seriously?

As Jennifer Marohasy says, without actually saying so the BOM admits the skeptics were right.

The BOM wants to stop this sort of error being discovered

For me the absolute red-flag, radioactive recommendation is this one where the Panel recommends changing their website in a way that would hide the exact inconsistencies that make this public error detection possible. They want a less complicated BOM reporting system — saying that currently it is possible for different temperatures for the same site/time to be on the Internet in public:

“The Review Panel found that: … the current data flow architecture creates situations where data can be delivered to, and displayed on, the Bureau’s website via multiple pathways and this can be potentially inconsistent and confusing for end users;

Recommendation 6: Future investment in supporting IT systems should, as part of their design and system architecture, streamline and improve efficiency and consistency in data flows.”  — page 12 of the PDF.

The review panel didn’t thank the citizen scientists who helped them find an error the experts had missed for years by noticing the inconsistencies in the live and long term data streams. Instead the BOM’s priority is to not get caught again, by rejigging the system to get “consistency”. What matters more: accuracy, error detection, or “consistency”? It depends on whether you are a scientific unit or a PR unit.

The BOM review tries to palm off the citizen scientists who were right, and more careful than them, as “confused”. In this in-house review the million-dollars-a-day BOM proves beyond a doubt that their highest priority is to protect their own jobs, not to collect accurate information about the Australian climate.

Thredbo

Thanks to Bob Fernley-Jones for graphing the data from Thredbo. The maxima in red are at the top. The minima in blue below. In green the data recorded on the new electronic thermometer which was faulty for seven years (which is not even the same faults we are discussing in the review.) Between 1997-2004 the electronic thermometers were rounding temperatures to whole degrees. So much for the 0.2C accuracy. On the minima side, it is obvious to the eye that since electronic thermometers were installed there have been no temperatures below minus 11 (I thought we weren’t supposed to even get below minus 10.4?).

Perhaps this was a climate shift that occurred around the same time the electronic equipment was brought in? Perhaps it wasn’t. Where is the raw side-by-side data of the two year overlap between old and new equipment? There have been a lot of -10.4Cs since the flawed MS1 hardware card was installed in 2007.

Thredbo, maxima, minima, 1966 – 2017. Click to enlarge.  |  Graph: Bob Fernley-Jones.

I’m not suggesting that a few truncated minima, if that’s all it was, necessarily affect our long term trends. (Though it may affect press releases about cold records). Nor am I suggesting it was deliberate. The bigger issue for me, the reason this matters, is yet again we see what kind of scientific standards and attitude are behind the work the BOM do. Accidents happen, but lack of interest in error detection and correction and the cover-up’s are not unwitting. Do the details of the Australian climate matter to the BOM?

Trust us? Are you kidding?

If we could trust the Bureau of Meteorology, the latest report would still be stretching things. But trust is something the BOM burned at the stake years ago. The list is too long. For starters, what trustworthy group avoids any and every independent audit? In 2012 they threw out their supposedly High Quality data set (it was actually called that) as soon as a serious audit was called. This is an organisation that retrospectively adjusts data in the past with a pattern of a non-random cooling bias, and is strangely uninterested in hot historic high temperatures recorded in Stevenson Screens before 1910. It’s a supposedly scientific group that hides their methods from the public. The same BOM admits it uses one-second random noise for high side records, while “not noticing” for years that faulty equipment was accidentally deleting low side records and auto-replacing them with warmer numbers.

Here’s a few choice picks of BOM fails — the highest temperature ever recorded in their Hallowed ACORN data set occurred in Albany far south WA, but only after adjustments. This basic error quality control error is a flag for who knows how many other unexplained changes, but even this radioactive “hottest day ever recorded in Australia” hasn’t been corrected. (Three years and still counting.) Who cares about good data when it’s only the planet at stake?

The BOM homogenize stations a thousand kilometers apart, they use city stations, bad stations, to adjust the good ones by a mysterious unpublishable process. Their methods generated a thousand days where mimima were absurdly higher than maxima. They adjust temperatures up-down-up on a calendar month with major corrections whipping up and down 2 degrees overnight that defies any kind of meteorological explanation. They introduced a new electronic thermometer system right across Australia in the 1990s which coincides with a jump up in temperatures. They say they carefully calibrate the two systems, but they’ve deleted all the side by side raw data, so who knows? Who will ever know?

The Australian climate data set is possibly beyond recovery — damaged beyond repair. We’re at the point where if Frydenberg and Turnbull won’t do something to serve Australian citizens and the Australian environment we need to set up our own independent stations. Someone needs to collect data that the Australian people can trust. We need thermometers outside the control of the BOM.

The current review is full of “confidence-building” but totally unjustified language:

This includes taking a highly precautionary approach and ensuring any location that has recorded below -5 degrees Celsius in the past has equipment capable of recording down to -25 degrees Celsius.

The previous approach was sloppy, lazy, inept and for years, and would not have been discovered without volunteers. Now they discover the highly precautionary approach?

What kind of “rigorous” control includes not publishing the full methods or data?

Page 12 “The controls around ACORN-SAT are particularly rigorous, requiring a minimum two-year overlap between sites or systems when observing equipment is changed or relocated, or techniques are changed. System changes, and any other events that might have an impact on data records, are documented in the Bureau’s station metadata repository SitesDB, in accordance with WMO requirements.”

Question 1: Is the SITESDB open to the public?

Question 2: Does it contain raw data from the overlap period?

There is so much more to say about this, and the many PR terms in what is supposed to be a scientific reply, but for the moment, why trust the BOM?

The BOM review

Bureau has a budget of $365 million a year.

 BACKGROUND — Scandal after scandal

Keep reading  →