Matt Ridley looks at pharmaceutical research and finds problems of confirmation bias, lack of access to data, and lack of replication of results. He compares it to the hockey stick debacle which is rising in notoriety to become the new benchmark of bad science. Articles like this are especially useful, because people concerned about Tamiflu might not know anything about the HockeyStick, and might not have read an article about the climate.
In Pharmaceutical research companies may do many studies on a drug but only choose to publish the ones with results they feel better about.
PERHAPS it should be called Tamiflugate. Yet the doubts reported by Britain’s House of Commons public accounts committee last week go well beyond the possible waste of nearly half a billion pounds ($913 million) on a flu drug that might not be much better than paracetamol. All sorts of science are contaminated with the problem of cherry-picked data.
Science at a breaking point:
The problem seems widespread. A paper in the BMJ in 2012 reported that only one fifth of clinical trials financed by the US National Institutes of Health released summaries of their results within the required one year of completion and one third were still unpublished after 51 months.
The legendary bad hockey-stick saga is related to a new audience:
To illustrate how far this problem reaches, a few years ago there was a scientific scandal with remarkable similarities, in respect of the non-publishing of negative data, to the Tamiflu scandal. A relentless, independent scientific auditor in Canada named Stephen McIntyre grew suspicious of a graph being promoted by governments to portray today’s global temperatures as warming far faster than any in the past 1400 years – the famous “hockey stick” graph. When he dug into the data behind the graph, to the fury of its authors, especially Michael Mann, he found not only problems with the data and the analysis of it but a whole directory of results labelled “CENSORED”.
This proved to contain five calculations of what the graph would have looked like without any tree-ring samples from bristlecone pine trees. None of the five graphs showed a hockey-stick upturn in the late 20th century: “This shows about as vividly as one could imagine that the hockey stick is made out of bristlecone pine,” wrote McIntyre drily. (The bristlecone pine was well known to have grown larger tree rings in recent years for non-climate reasons: goats tearing the bark, which regrew rapidly, and extra carbon dioxide making trees grow faster.)
McIntyre later unearthed the same problem when the hockey-stick graph was relaunched to overcome his critique, with Siberian larch trees instead of bristlecones. This time the lead author, Keith Briffa, of the University of East Anglia, had used only a small sample of 12 larch trees for recent years, ignoring a much larger data set of the same age from the same region. If the analysis was repeated with all the larch trees there was no hockey-stick shape to the graph. Explanations for the omission were unconvincing.
One of the comi-tragic ironies here is that the scientist who “did the most” to expose the Tamiflu story according to Ridley is Ben Goldacre — but is this the same Ben Goldacre who appeared on the ABC propaganda-doco about climate (with Nick Minchin and Anna Rose, and which myself and David were also involved in). In that doco though, Ben Goldacre talks of denialists, and trust and how he trusts the experts on climate change:
Ben Goldacre: So it’s not that I trust them because I think they’re nice people or that I think they sort of play with a straight bat generally, like I’m basically assuming people aren’t actively lying, when I trust somebody else’s scientific opinion, when I trust the majority of opinion in a whole field, it’s because I know from all the work that I’ve done in other fields that there are checks and balances and structures where people will critique each other’s ideas and they will pull out the killer refutation of somebody else’s claim…. I’ve got no reason to believe that’s not happening just as healthily in climate research as it is anywhere else. ["I can change your mind" transcript]
Goldacre continued with this memorable gem:
Nick Michin: Have you read Booker’s book?
Ben Goldacre: Of course I haven’t, you know, these people are idiots. Chris Booker says that what is it, he’s got some bee in his bonnet about how asbestos isn’t really bad for you. You know, I mean, these are – they’re just not very interesting to people to most people, you know. If you’re really, you know, if
you’re really into climate change denialism then I’m sure this guy is like a massive figure to you but it’s just boring, it’s a boring, boring argument. I would literally rather slam my cock in the door than get involved in this.
So Ben would rather slam his **** in a door than get involved in this climate denial argument, however appearing in a documentary and staking his reputation on it, that’s different. He didn’t need to do any research for that eh? He said as much:
Ben Goldacre: And so I, rightly or wrongly, just kind of gave up… To have a big argument, meaningfully with the climate change deniers I would have to familiarise myself with this vast body of evidence and actually I don’t think it will be sufficiently good fun but I can’t be bothered, you know.
Message to Ben: no one said you had to go on TV to talk about climate science. If you can’t be bothered doing any research, just say “No, thanks”. Crass ignorance is not the best look.
Poking holes in Big-Pharma, when they deserve it, is all admirable and worthy but it’s an Occupy-Science kind of approved target. Real scientists and investigators who unpick dodgy science claims are the ones who will tackle any field of science, not just the ones on the fashionable-hit-list. After all, those who attack the critics of bad-science and defend the bad-scientists are pretty bad-scientists themselves. Touche?
Ben is welcome to join us in the real trenches, but he will need to do some reading first.