Earlier this week, Frontiers in Public Health published the abstract of a paper called ‘Vaccination and Health Outcomes: A Survey of 6- to 12-year-old Vaccinated and Unvaccinated Children based on Mothers’ Reports’. Based on an online survey of 415 mothers involved in the homeschool movement, Mississippi-based researchers Mawson et al. reported that vaccination is associated with a much higher rate of neurodevelopmental disorders, including autism.
The Mawson et al. paper led to a lot of controversy, not least on Twitter. On Monday, many people, myself included, tweeted concern over seeing such a piece in a peer-reviewed journal. Frontiers, the Swiss publisher of the journal in question, took to Twitter to say that the article “was provisionally accepted but not published” and that “In response to concerns raised, we have reopened its review.” Minutes later, the paper disappeared, and if you visit its URL now, you will find nothing but an error message. (Here’s a copy, though.)
I don’t think it’s either, really, but this case does raise interesting questions about how we judge science. Is it right to object to a paper just because its results fly in the face of most previous research?
Everyone agrees that it is fair to critique a study on the basis of the methods. And many people did criticize the methodology of the Mawson et al. study, pointing to serious problems such as the small sample size (relative to the huge studies showing vaccines are safe), the purely self-report measures, and the potential for recall and selection bias.
Yet I don’t think that so many people would have been so critical of Mawson et al.’s methods if it weren’t for the nature of their findings. Studies suffering from the same flaws, or worse, get published all the time across many fields. Twitter doesn’t explode over every bad study. So isn’t there a risk that scientists are selectively sceptical, scrutinizing studies that challenge the consensus?
On the other hand, it’s true that the scientific consensus exists for a reason. As I said in one of my first-ever posts, we should beware the myth of the Galileo-like lone scientist who turns out to be right while everyone else is wrong:
All of our most popular myths about science are Robin Hood stories – the hero is the underdog, the rebel, the maverick who stands up to authority… the hero is a denialist. Once, this was realistic. Galileo was an Aristotelean cosmology denier; Pasteur was a miasma theory denier; Einstein was a Newtonian physics denier. But these stories are out of date… Science has moved on since the time of Galileo, thanks to his efforts and those of they who came after him, but he is still invoked as a hero by those who deny scientific truth. He would be turning in his grave, in the earth which, as we now know, turns around the sun.
In fact, it’s fair to say that if we were to reject everything that challenges the scientific consensus, we would be right to reject them in the vast majority of cases. But however accurate the consensus is, science is not supposed to be a matter of consensus, but a process of observing the world. The only thing that should matter, in judging science, is the quality of those observations, i.e. the strength of the methodology.