José Duarte is a psychology PhD candidate. He is able to make sense of issues in the “Moon Landing Paper” by Stephan Lewandowsky, with some new angles in a way I haven’t seen before. He makes a convincing case for the paper to be retracted, about six times over. My initial analysis of this paper still stands: “This could be the worst paper I have seen — an ad hom argument taken to its absurd extreme, rebadged as “science”.
Duarte focuses on the deception of a title based on only 10 responses, some of which were fakes, none of which was disclosed to the reader:
Lewandowsky, Oberauer, Gignac titled their paper “NASA faked the moon landing—therefore, (climate) science is a hoax: an anatomy of the motivated rejection of science.”
Why is their title based on the variable for which they have the least data, essentially no data?
Why in the abstract are they linking free market views to incredibly damaging positions that again, they have no data for?
Could this be an error? That seems very unlikely. The researchers have had two years to come clean, to admit that there was no significant data regarding belief in the moon hoax or rejection of the HIV-AIDS or smoking-lung cancer links. They’ve had two years to remove the very-likely-to-be-scam participants identitifed by people who have looked at the data, which will further reduce those trivial numbers at the bottom, and they’ve not done so. I’m not sure they even talk about it. Lewandowsky still won’t tell the public that fewer than 10 participants rejected the moon hoax or HIV and smoking claims – after all this time Lewandowsky is still evading those basic facts and distracting his readers with nonsense statistics. Pearson correlations on essentially dichotomous data skewed 1135 to 10? The paper should have been retracted by the authors long ago.
Duarte reminds us that the title contains a form of reasoning, a causal directional claim for which there is no data. As if readers start from deciding the Moon landing was faked then use that “therefore” to reject “climate science”. In the end there were four (count them, 4)* anonymous online respondents who said the Moon landing was faked AND that climate science was a hoax. Why won’t Psychological Science retract this paper?
The data was so bad, Duarte calls it “go home” data.
I don’t know what’s going on at Psych Science – the stats here were amateurish and deceptive. First, the data here was go home data. If you want to link moon hoax nonsense to your political foes, and in 1145 participants there are only 10 people who endorse that hoax (fewer after you delete the fakes), only 3 of whom endorse the climate hoax idea (fewer after you delete the fakes), you go home. It’s over. If you see similar trivial numbers for the HIV and smoking items, you bail. Go to a show, discover a new restaurant, think about the design of your next study. Those are go home numbers – you definitely don’t write it up.
Duarte asks “Do we hate our participants?”
This was an awful thing to do. It was damaging to innocent participants. It’s unethical to do this to your participants. It is wildly unethical to invite people to participate in a study, and then do this to them. They are helping us. They are volunteering to participate in scientific research. They’ve take time out of their lives to help us out. And in return, we slander them? We tell the world that they believe things that they do not believe? What Lewandowsky and colleagues did here was despicable.
Why would anyone participate in our research if our goal is to marginalize them in public life, to lie about them, to say that they think the moon landing was a hoax, to say they don’t think HIV causes AIDS, to say they don’t believe smoking causes lung cancer – when none of those things are true.
Simon Turnill also asked something similar – pointing out that the Australian National Statement of Ethical Conduct in Human Research (here) says “one of the duties of a researcher is to “ensure that respect for the participants is not compromised by the aims of the research”.
Duarte wonders why Lewandowsky et al chose to ignore associations which had more data (some data) to discuss, and why they didn’t even report some results at all (see the “lost” questions here), yet chose to headline invalid associations based on ” invalid associations based on 10 or 11 participants”.
When they wrote their title, they had a list of at least 13 conspiracy beliefs in their data, things that might be related to climate skepticism or free market views. They chose to talk about the conspiracy that had the lowest endorsement of everything in their dataset, so low that it’s not even there. And most of those not-even-there moon hoaxists, 7 out of the 10, did not endorse a climate hoax (if we even want to talk about such trivial numbers, which we’re surely going to decide we don’t.) And it’s such a damaging conspiracy to be associated with.
When you have no data, you tell no story. If you’re talking about things that are incredibly damaging, beliefs and positions which would marginalize people in public life, you definitely tell no story when you have no data.
Duarte explains that the four point scale Lewandowsky used was really a binary one, not a continuous scale. When treated as “continuous” spurious inane results get generated. Where a group that was more likely to say they “disagreed” with something, rather than “strongly disagreed” could be misconstrued as being more likely to “agree” with a conclusion that they actually said they disagreed with.
The disconnect with reality and real denial occurs with the social psychologists, not the participants:
If we wanted to identify the people disconnected from reality in this picture, it is the social psychologists, the reviewers, the journal editors who read passages like “Endorsement of free markets also predicted the rejection of other established scientific findings, such as the facts that HIV causes AIDS and that smoking causes lung cancer” and didn’t stop and wonder at the plausibility of such a thing, who evidently thought lots of people reject the HIV-AIDS link, or lots of people reject the smoking-cancer link, and that these beliefs go with endorsing free markets. They read this title: “NASA faked the moon landing—therefore, (climate) science is a hoax: an anatomy of the motivated rejection of science”, and presumably thought this was plausible, that a lot of people think the moon landing is a hoax, and that this was the springboard for climate skeptics (or hoaxists). If I wanted to talk about disconnect from reality, “denial” and the like, I wouldn’t focus on the participants.
He makes a convincing case that with tiny response rates, not only are some faked, but other are simply errors of people hitting the wrong button, misreading the question or mixing up terms.
Let’s even say someone is aware of the link between HIV and AIDS. I would bet that in a large sample, you’re going to find some people who think AIDS causes HIV. That’s very easy to imagine — they have the link, but somehow got the direction wrong. Some people might see “HIV causes AIDS” and think it’s a trick question with the wrong order, and thus submit their disagreement.
I found the analysis of new aspects of the paper interesting but would understand people who ask if there is any point is spending more time analyzing any survey, or treating any of its results as worthy of discussion, given that it started from the basis of trying to “discover” what skeptics think by asking people who hate skeptics. That was my first post on the topic: Lewandowsky – Shows “skeptics” are nutters by asking alarmists to fill out survey. The hostile aims of the survey were so transparent that even anti-skeptic commenters were calling it out:
“Yeah, those conspiracy theory questions were pretty funny, but does anyone think that hardcore deniers are going to be fooled by such a transparent attempt to paint them as paranoids?” pointer | August 30, 2010 at 11:42 am
Nonetheless, I’ve learned things about survey analysis from Duarte, which I enjoyed. He’s a lucid writer.
h/t to Bishop Hill.
- Lewandowsky, Cook claim 78,000 skeptics could see conspiracy survey at Cooks site where there is no link
- Journal admits Lewandowsky paper retracted because it failed. Twice!
- Lewandowsky gets $1.7m of taxpayer funds to denigrate people who disagree with him
- Ask Stephan Lewandowsky anything, except “please can I have that data”.
- Lewandowsky — A paper of questionable ethics, approved in a last minute switch
- Lewandowsky does “science” by taunts and attempted parody instead of answering questions
- Steve McIntyre finds Lewandowsky’s paper is a “landmark of junk science”
*Pointless technical question: were there 3 or 4 of 10 who said yes to the Moon landing conspiracy who also said yes to the climate change “hoax” one? The first pivot table here (and my spreadsheet) suggest 4, but others say 3. It’s not like it makes any difference.