Richard Tol has an excellent summary of the state of the 97% claim by John Cook et al, published in The Australian today.
It becomes exhausting to just list the errors.
Don’t ask how bad a paper has to be to get retracted. Ask how bad it has to be to get published.
As Tol explains, the Cook et al paper used an unrepresentative sample, can’t be replicated, and leaves out many useful papers. The study was done by biased observers who disagreed with each other a third of the time, and disagree with the authors of those papers nearly two-thirds of the time. About 75% of the papers in the study were irrelevant in the first place, with nothing to say about the subject matter. Technically, we could call them “padding”. Cook himself has admitted data quality is low. He refused to release all his data, and even threatened legal action to hide it. (The university claimed it would breach a confidentiality agreement. But in reality, there was no agreement to breach.) As it happens, the data ended up being public anyhow. Tol refers to an “alleged hacker” but, my understanding is that no hack took place, and the [...]
How to separate creative genius from creative mistakes? Not with peer-review. It is a consensus filter.
Classical peer review is a form of scientific gatekeeping (it’s good to see that term recognized in official literature). Unpaid anonymous peer review is useful at filtering out some low quality papers, it is also effective at blocking the controversial ones which later go on to be accepted elsewhere and become cited many times, the paradigm changers.
And the more controversial the topic, presumably, the worse the bias is. What chance would anyone have of getting published if, hypothetically, they found a consequential mathematical error underlying the theory of man-made global warming? Which editors would be brave enough to even send it out for review and risk being called a “denier”? Humans are gregarious social beings, and being in with the herd affects your financial rewards, as well as your social standing. Even high ranking science journal editors are afraid of being called names.
Mark Peplow discusses a new PNAS paper in Nature:
Using subsequent citations as a proxy for quality, the team found that the journals were good at weeding out dross and publishing solid research. But they failed — quite spectacularly — [...]
If you suffer from an uncontrollable urge to claim that peer review is a part of The Scientific Method (that’s you Matthew Bailes, Pro VC of Swinburne), the bad news just keeps on coming. Now, we can add the terms “Peer Review Rigging” to “Peer-review tampering”, and “Citation Rings”.
Not only do personal biases and self-serving interests mean good papers are slowed for years and rejected for inane reasons, but gibberish gets published, and in some fields most results can’t be replicated. Now we find (is anyone surprised?) that some authors are even reviewing their own work. It’s called Peer-Review-Rigging. When the editor asks for suggestions of reviewers, you provide pseudonyms and bogus emails. The editor sends the review to a gmail type address, you pick it up, and voila, you can pretend to be an independent reviewer.
One researcher, Hyung-In Moon, was doing this to review his own submissions. He was caught because he sent the reviews back in less than 24 hours. Presumably if he’d waiting a week, no one would have noticed.
Nature reports: “THE PEER-REVIEW SCAM”
Authors: Cat Ferguson, Adam Marcus and Ivan Oransky are the staff writer and two co-founders, respectively, of Retraction Watch in [...]
An excellent article in The New Yorker: Is Social Psychology Biased Against Republicans?
It’s an article about the failings of peer review and research design in psychology due to the dominance of one particular political ideology (rather than having a spread more representative of the total population). You won’t be shocked to find there is a dominance of liberal left-leaning views in the profession. The paper it discusses is by Jonathan Haidt and co-authored by our friend Jose Duarte — the psychology PhD candidate and blogger who entertainingly and comprehensively dissected Lewandowsky on his blog: Do we hate our participants?
It will be no surprise that controversial psychology papers (which disagree with the reviewer’s world view) are usually treated harshly — no matter if the data is as strong. So, thinking of another field we know, what does it mean for research design and peer review when 97% of certified climate scientists hold one world view? (They not only agree on the scientific hypothesis but on the political action as well — and they boast about that?) What chance does a “controversial” paper have? Has anyone done a study on the political diversity of official climate scientists? There are plenty [...]
Most of the results reported in peer reviewed literature in medicine are mere artefacts of poor methodology, despite being done to more exacting standards than climate studies. There are calls in the medical literature for all data to be made public and for higher P values to be required. (Yes please say skeptics everywhere). Miller and Young recommend that observational studies don’t be taken at all seriously until they are replicated at least once. That would have ruled out the original HockeyStick two times over.
Even the absolute best medical papers are wrong 20% of the time, but mere observational studies (like climate research) failed 80 – 100% of the time. These studies of papers demonstrate why anyone who waves the “Peer Review” red flag is in denial of the evidence — “Peer Review” is not part of the scientific method. It’s a form of argument from authority. A fallacy of reasoning is still a fallacy, no matter how many times it is repeated. Those who claim it is essential or rigorous are not scientists, no matter what their government-given title says.
GEN, Genetic Engineering and Biotechnology News, May 1, 2014, Point of View
Are Medical Articles True on Health, [...]
Sir Isaac Newton
Peer review by anonymous unpaid reviewers is not a part of the Scientific Method.
Once upon a time the fate of a scientific paper was dependent on an Editor whose reputation depended on making sound decisions about what to publish. Modern science shifted responsibility from a single identifiable editor to an anonymous “committee”. What could possibly go wrong?
From Zocalo Public Square
Melinda Baldwin looked at the history of peer review:
I was incredibly surprised to learn that Nature published some papers without peer review up until 1973. In fact, many of the most influential texts in the history of science were never put through the peer review process, including Isaac Newton’s 1687 Principia Mathematica, Albert Einstein’s 1905 paper on relativity, and James Watson and Francis Crick’s 1953 Nature paper on the structure of DNA.
A revolution in science happened without formal “peer review”. Who would have thought?
Crucially, journals without refereeing processes were not seen as inferior or less “scientific” than those that used referees. Few scientists thought that two anonymous readers would better judge a paper than, say, the great physicist Max Planck (who was on the editorial board of the prominent German journal [...]
At least 120 computer generated nonsense papers have been reviewed and published in publications of the Institute of Electrical and Electronic Engineers (IEEE) and Springer, as well as conference proceedings. The fakes have just been discovered by a French researcher and are being withdrawn.
Cyril Labbé found a way to spot artificially-generated science papers, and published it his website and lo, the fakes turned up en masse. In the past, pretend papers have turned up in open access journals–this time the fake papers appeared in subscription based journals. But the man who caught the fakes says he cannot be sure he’s caught them all, because he couldn’t check all the papers behind paywalls.
According to Nature:
The publishers Springer and IEEE are removing more than 120 papers from their subscription services after a French researcher discovered that the works were computer-generated nonsense.
Over the past two years, computer scientist Cyril Labbé of Joseph Fourier University in Grenoble, France, has catalogued computer-generated papers that made it into more than 30 published conference proceedings between 2008 and 2013. Sixteen appeared in publications by Springer, which is headquartered in Heidelberg, Germany, and more than 100 were published by the Institute of Electrical and [...]
Over a week ago Christopher Monckton sent this letter below to the Editor of Copernicus Publications, suggesting they reconsider their hasty decision to close the journal, and informed Martin Rassmussen that unless he heard from him about that or about copyright issues within 7 days, Monckton would take over the title Pattern Recognition in Physics and relaunch the journal. There was no response from Copernicus, so Monckton is now free to pursue this. I think it is a good development, and hope it will lead to a dispassionate discussion of the scientific ideas that were raised.
The scandal remains that Copernicus did not close the journal because of any scientific flaws. They first and foremost closed the journal because it “doubted” the IPCC, as they baldly declared in their original emails and official statement. That Copernicus then post hoc claimed there was a fault with the reviewing process doesn’t change the fact that a major scientific publishing house took the extraordinary decision that the IPCC can not be questioned and naively admitted it, as if it was acceptable. It reveals the utterly unscientific mindset of the gatekeepers of Peer Review.
My position is that Peer Review is a bureaucratic process [...]
The termination of the Pattern Recognition journal ought to be PR gold for skeptics. Nothing like this happens to unskeptical scientists, ever. It’s a telling spectacle.
A major science publisher gasped in horror that one amongst its scores of journals had “doubted the IPCC”, so the journal had to be axed forever. Oh the Crime! The over-the-top dummy-spit exposes the religious zeal in a supposedly scientific process. So much for the hallowed “Peer Review”. Fans of establishment science want us to believe it’s a gospel part of the scientific method, but it is neither intrinsic nor essential, and skeptics should not be fooled into thinking it is.
Peer reviewed papers may be gems or junk but we won’t know which by discussing who reviewed them.
Let’s follow due process in science, but that is not by review whether peer-or-pal, it’s by prediction, test, observation, and repeat.
Some people seem to have lost sight of this, and think that skeptics ought to be trying to play the Peer Review Game according to the fine print of arbitrary rules dictated by unscientists who hate skeptics and who don’t even play by the rules themselves. The game belongs to them – they set, [...]
In extraordinary news, the scientific journal Pattern Recognition in Physics has been unexpectedly terminated, a “drastic decision” taken just ten months after it started.
The publisher appears to be shocked that in a recent special issue the scientists expressed doubt about the accelerated warming predicted by the IPCC. For the crime of not bowing before the sacred tabernacle, apparently the publishers suddenly felt the need to distance themselves, and in the most over-the-top way. The reasons they gave had nothing to do with the data, the logic, and they cite no errors. There can be no mistake, this is about enforcing a permitted line of thought.
I must say, it’s a brilliant (if a tad expensive) way to draw attention to a scientific paper. It’s the Barbara-Streisland moment in science. Forget “withdrawn”, forget “retracted”, the new line in the sand is to write a paper so hot they have to terminate the whole journal! Skeptics could hardly come up with a more electric publicity campaign.
Naturally, as with all good Barbara-Streisland-moves intended to suppress information, as soon as I heard, the first thing I did was to seek out and download copies of all the papers. Right now, people [...]
17 contributors have published
1828 posts that generated