Science is broken. The genius, the creative art of scientific discovery, has been squeezed into a square box, sieved through grant applications, citation indexes, and journal rankings, then whatever was left gets crushed through the press. We tried to capture the spirit of discovery in a bureaucratic formula, but have strangled it instead.
There are no shortcuts to the truth, or to status, and no easy way to figure out which projects should be funded. Every time a decision is crowd sourced — via committee, panel, or “consensus” — the responsibility for thinking gets divided and avoided.
The modern bureaucratic process of science is now not even trying to search for the truth. It’s hunting instead for an impact factor, for attention, for headlines, and inevitably, for funding.
It is good to see people starting to discuss it — including the Lancet Editor, Richard Horton, who wrote in April that he could not name names, but it needed to be said:
“A lot of what is published is incorrect.” I’m not allowed to say who made this remark because we were asked
to observe Chatham House rules. Those who worked for government agencies pleaded that their comments especially remain unquoted…
…[it is] one of the most sensitive issues in science today: the idea that something has gone fundamentally wrong with one of our greatest human creations.
The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue.
Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts
of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has
taken a turn towards darkness. As one participant put it, “poor methods get results”.
Richard Horton is talking mostly about biomedicine, but the problem is endemic:
Part of the problem is that no-one is incentivised to be right. Instead, scientists are incentivised to be productive and innovative.
More red tape won’t set science free
I don’t think his suggestions are the answer, and even Horton seems to agree with that. A Hippocratic Oath for science, will help, but not much. Similarly, writing regulations to insist on a certain percentage of replicability in grant applications is only tinkering at the edges. As is emphasizing collaboration rather than competition, or insisting on “preregistration of protocols”. Likewise, rewarding “better pre and post publication peer review”, or improving research “training and mentorship”. None of that will make discovering the truth the main game again.
Lets start the list of what we need
What we need (for starters) is better training in logic and reason, and it needs to start in primary school. All kids need to know what an ad hominem argument is, and to spot the weak argument from authority. I shouldn’t need to explain what those are to a science graduate, a science communicator, a science journalist, or a science minister. A professor who can’t reason, shouldn’t be a professor. Actually I shouldn’t need to explain these fallacies even to a 12 year old, because it should be rote learned by 10.
Then we need to fix the incentives. We need to find a way to reward creative genius which breaks assumptions, rather than the sort that just fits in the box. We need to let genius flourish again, instead of bureaucracy.
To fix science we also need to fix science journalism, and science communication. Because these ought be another layer of protection. Good journalists and interviewers shouldn’t let scientists get away with dumb answers. Good science communicators serve the public, not the bureaucratic science-machine. Instead our supposedly best science magazines just report smear by association: see New Scientist: The Age of Name-Calling.
Posts on logic and reason.
H/t Steve (2 weeks ago), Catallaxy, David, Jim, Willie.