The Problem of Selective Skepticism

By Neuroskeptic | November 22, 2015 9:32 am

What does it mean to be a skeptic? Can one be skeptical about one thing, and a true believer in something else? Or is selective skepticism not really skepticism at all?

dreamsThis is an issue that has been worrying me.

As my pseudonym suggests, I try to treat neuroscience skeptically. I try my best to think seriously about claims made about the brain and check them against the facts, the evidence. This doesn’t mean I reject everything: it means I accept only what the evidence supports.

So why would a skeptic be worried? Well, my worry is that I might be being selectively skeptical. Am I only skeptical about certain types of claims? If so, is that a problem? More broadly, there are plenty of people out there who appear to be selective skeptics. My question is, does that invalidate their ‘skepticism’?

My answer is that a selective skeptic is still a skeptic, but a bad one.

There are two different frames of reference here. If we focus narrowly on a particular claim, then if I’m skeptical about that claim, then I’m either right or I’m wrong, and the only thing that matters is the evidence in this particular case. In other words, if I were, say, a creationist who rejects all evolutionary science, I might still be right when I express skepticism about a particular evolutionary claim. And not just in the sense that even a stopped clock is right twice a day: I might genuinely uncover a problem with the argument.

However, while a creationist “evolution skeptic” could be a genuine skeptic in a narrow sense, this selective skepticism is clearly not desirable.

Suppose that I were to look skeptically at the claims made by one side in a certain scientific or public debate. I scrutinize their arguments and I find various problems; but for whatever reason, I never look into the claims coming from the other side of  the controversy. In this scenario, I would have become, in effect, a partisan for the side I didn’t criticize – even if this wasn’t my intention. My selective application of criticism would have made me an uncritical defender of one side.

Now in this scenario, I think I would still be a skeptic, but I’d be a bad one, because I am (so to speak) using my skeptical powers for evil. Instead if seeking truth wherever it is to be found, I would be digging up dirt and only flinging it one direction.

ADVERTISEMENT
  • PsychBrief

    I think it’s incredibly difficult to be a genuine sceptic (unless you’re a Pyrrhonian sceptic but that’s exhausting and makes getting anything done difficult). There are always going to be times where you sceptical hat slips slightly, when you don’t analyse a claim with the rigour you should. I would agree it makes you a “bad sceptic” but I think we should recognise most of us slip up and we should try and monitor ourselves to make sure we are suitably sceptical (though of course that’s easier said than done). You could remind others to be more sceptical but they might not appreciate it

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      I agree. Certainly no-one can be a perfect skeptic, we are all human.

      • paul

        What do you mean by we are all human? Why a human cannot be a perfect skeptic? Is this because we are all imperfect?

      • C’est la même

        “we are all human”

        Are you sure?

  • storkchen

    Great post! I think that potential selective skepticism plays an even bigger role in issues with political salience, like gender, climate, IQ, etc.

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      Right – to the extent that skepticism can get a bad name, and be seen as a cloak for contrarianism. My point in this post is that we can’t just reject skeptical arguments, even if the skeptics are clearly biased; we have to check the evidence. Biased skeptics are a bad thing, but, unfortunately, they may sometimes be right.

      • arch 1

        We are all biased. That is a good thing if based in fact,
        I have a strong bias against leaping off tall buildings expecting to fly or putting arsenic on my cornflakes.
        There is a place for right or wrong, good or bad.

  • Dan Riley

    Selective skepticism is a kind of bias, and we are all biased to some degree. Good skeptics should be aware of and skeptical of their own biases.

  • feloniousgrammar

    The term “skepticism” is often used by the paranoid and conspiratorial to describe themselves, now. Being judicious, discerning, and scrutinizing information or perspectives are practices that are only as good as the actual practice, skills, knowledge, and the appropriate use of such thinking. It also requires self-awareness, case specific knowledge, and thinking about one’s thinking in order to challenge one’s own biases.

    Seems to me that being sceptical is not something a person can be, in a global sense. It’s something one needs to DO conscientiously, so questioning one’s own sceptical abilities seems a natural and healthy part of “being a sceptic”.

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      “The term “skepticism” is often used by the paranoid and conspiratorial to describe themselves, now.”

      Indeed, and that’s what motivated this post.

      • feloniousgrammar

        Didn’t there used to be an edit option, here?

        • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

          An edit option for comments? Yes, that’s still there for me.

  • Not this again

    Some of us have been wondering for a long time how it is that so-called ‘skeptics’ have chosen to look at various aspects of Myalgic Encephalomyelitis (call it Chronic Fatigue Syndrome if you must) and make lots of noise about only those aspects that they deem worthy of comment, but selectively, as you say, leave plenty of things out.

    I suppose it was always easy to cast a dim view on ME/CFS–what most people knew of it was that it was based on subjective complaints, little of which corresponded with test results which typical show a whole lot of normal. If anyone had been willing to do, in the past, a little digging, they might’ve seen a connection to Royal Free, but it didn’t happen. So instead of extreme weakness, immunological issues, and severe neurological issues, it was about ‘fatigue’ and vague complaints of ‘pain’ like ‘joint aches’ or ‘headaches’ or ‘sore throats.’

    Skeptics are real, real down on alt-med, needless to say. The USA’s National Center for Complimentary & Alternative Medicine is a frequent target. Every now & then Stephen Straus, who co-founded that place, gets a dishonorable mention for sowing the seeds of this fraud, this gobbler of taxpayer dollars.

    Yet Stephen Straus, the person possibly most responsible for the modern dim view of “CFS,” never seems to get the same treatment from ‘skeptics’ when it comes to the other large item in his legacy, one he worked very hard on, for a long, long time.

    Any observation of the history of CFS is littered with red meat for skeptics, perhaps none more so than XMRV and Judy Mikovits. Fair enough; there were errors, it wasn’t handled well, looking unprofessional or worse to more than a few, and the whole affair ended under a very dark cloud. This made a lot of sense, considering how medical science tends to view CFS as somewhere between mis- or un-diagnosed depression, somatization, malingering, hypochondriasis, or some other sort of mental/emotional/behavioral issue, most likely psychosomatic. Medical schools are not known to spend much time on it, so it’s not like it’s widely considered to be an ‘organic’ disease with an origin that could be thought of as ‘physical’ (at least vs. ‘psychiatric,’ providing we’re avoiding another tedious romp through the Cartesian Dualism patch).

    So, when experts in the field publish a landmark study in the Lancet demonstrating that CFS patients improve from a course of behavioral interventions, why would any skeptic quarrel? This is what most people think, anyway, after all, that exercise is what these people need. And Cognitive Behavioral Therapy. What’s to argue with.

    When some people argued, they were mostly dismissed. Any and every argument put forth as to why this study was flawed was ignored as much as possible. This angered some people, but before long it’s revealed that the researchers in question have colleagues who have received death threats for their work in this area. There’s a host of headlines. What’s there to be skeptical about? After all, these people are just trying to help these patients. Why were they being questioned, anyway? They are without question the experts in the field, after all. Comparisons are made in the press to climate change denialism and even animal rights extremists. One of the most prominent psychiatrists involved with research in this area is quoted in the press as saying that he feels safer in war zones than he does in his office in London, at the Royal College.

    The patients who suggested these claims are at least exaggerated then watch that most prominent researcher first win a John Maddox Prize for his courage in the wake of these threats, and then is knighted by the British Government. Years pass. More papers follow from the landmark study, and, in spite of questions raised by critics (many of whom are patients), no ‘skeptic’ that I’ve ever seen takes any time to look at any of these claims. Plenty of ‘skeptical’ mentions of CFS here or there, the occasional blog, snarky comments on places like the James Randi forum and pieces that are just short of snarky, but no less typical, on sites like Kevin MD.

    A month ago a journalist who often writes about ME/CFS in the New York Times starts publishing a multi-installment critical analysis of the landmark study, which was known as PACE, and was labeled the ‘gold standard’ in the field. At this point a couple of people start to take notice; this analysis is on Virology Blog, which is not exactly a typical target of ‘skeptics.’ The following week, a new paper emerges from the PACE team. This time, though, they have published a paper that both demonstrates a null result, while proclaiming the opposite, as a follow-up study. The UK press, by now so used to rubber-stamping whatever comes out that reinforces the view that while this is a serious condition, it’s best handled by the experts, who have developed guidelines based on PACE and other research showing that the only treatments for CFS considered safe and effective are CBT and Graded Exercise Therapy.

    There have been a slow trickle of papers based on small studies from an underfunded group of researchers who show the opposite–that objective measures of the effects of exertion are notable and serious, and suggest that it’s the last thing that these patients should be doing. Indeed, earlier this year the US National Academy of Sciences’ medical arm published a review from a panel convened at the Institute of Medicine, that not only stated clearly that CBT and GET should not be considered safe or effective treatments, but that the name of the condition should be officially changed, to one that specifically points to exertion as being damaging.

    That study received a lot of press, at least in the US; but published the very same day was a Cochrane Review that found GET to be helpful in this condition. 5 of the 8 studies included in the Cochrane Review used the same CFS criteria used in the PACE study, the very important- and offiicial-sounding Oxford Criteria. The US National Institutes of Health had this year recommended that those criteria be retired as unfit for purpose, but who cares what they say. Certainly not ‘skeptics.’ Everybody knows the best treatments for CFS are CBT and GET.

    The stories and especially headlines in this current round of UK press following the latest PACE paper are far beyond any conclusion the researchers would likely have used: “Oxford University finds ME is not actually a chronic illness,” said the Telegraph. Others followed which probably don’t deserve to be repeated. But now there were people actually looking at this, given the analysis on Virology Blog, and finding that there were years of criticisms that had all been pretty much ignored. Some of those criticisms looked pretty valid now.

    The skeptic who deserves the credit for deciding that all of this was worth looking into is James Coyne. Based on his interest, Keith Laws weighed in. The most prominent researcher, Simon Wessely, published a defense of PACE on the Mental Elf website. Laws came forth with another blog that was critical of Wessely’s. Following an announcement from NIH Director Francis Collins that there would be a new approach, with expanded funding, available to study this condition, there was a barrage of coverage in the media, from the Atlantic to Slate, the Washington Post, Science, Mosaic, Gizmodo, the San Francisco Chronicle, and others.

    Not so much from the skeptics. Was any of this on any outlet of Science Blogs? Science-Based Medicine? Where was Ben Goldacre in this? One major complaint that the PACE team had was that they were being harassed by Freedom of Information requests for the raw data. All of which had been turned down, some labeled ‘vexatious.’ Ben Goldacre is one of those guys who spends a lot of time talking about how data should be made available, right? PACE was publicly funded, to boot. Heck, I don’t want to single him out–some of the people on the forum that used to be attached to his website once had a lot to say about how anyone who argued against PACE didn’t know what the heck they were talking about–and were quite rude, to say the least. Not much going on there, either. But there are more than a few names deserving of scorn for either writing nasty things about CFS over the years, or just focusing on whatever aspect of all of this was in their wheelhouse. It’s been a month since Virology Blog came out as critical of PACE. There’s been an open letter to the editor of the Lancet signed by several scientists. But, you know, the skeptics are on the job! If there’s someone out there working with CAM, or acupuncture, or chiropractic, they’re on it. Good work, you guys. Weil–bad! Mercola…bad! Integrative medicine…bad! CFS? Well, you know, the IOM says it’s a severely disabling disease characterized by intolerance to exertion, and their reaction was to make sure their readers knew it’s important to make sure these patients don’t melt into the ‘sick role.’

    Earlier this year when the IOM report came out there were a few doctors who went on television to talk about how the new name proposed for CFS–Systemic Exertion Intolerance Disease–made sense, since these patients cannot tolerate exertion. The next thing out of their mouths in some cases was that it’s also good to keep in mind that the research shows that CBT and GET are the best treatments, so it’s important for these patients to adhere to an exercise program.

    You can’t make this up. Wait. Did I say that before I pointed out that, again, there was NO ‘skeptic’ that found this remotely odd?

    But for years anyone who tried to appeal to a ‘skeptic’ that ME/CFS was something other than what most people think it is was basically treated like a young-earth creationist.

    I don’t have a quarrel with Neuroskeptic. I’ve enjoyed this blog over the years, and I think it might’ve actually even covered CFS once. But at some point, one has to ask: if you’re going to write a blog like this, why is what’s been going on with James Coyne and Keith Laws recently not worth mentioning? Did it escape your notice?

    Or is the omission just selective?

    • Christopher Lierle

      As someone whose life has been turned inside-out by this disease, I’m glad to see your comment/post. Just as the article says, skepticism is needed on BOTH sides, so that it is without agenda.

      I see this regarding one point in particular: the effectiveness of CBT. It is NOT an indicator that the illness is psychological in nature, even though that argument seems logical.

      Instead, CBT reduces biological exertion by helping a person learn to turn off unproductive psychological processes. To restrict the wasteful use one of the body’s leading consumers of energy, the brain, is to reduce exertion.

      There is an equally “common sense” explanation for this:
      If I teach you a brand new, complex technical concept for a couple of hours without a break, you will experience the “brain fog” of CFS/ME. Cutting the cognitive load (through best practices in instructional design) cuts the brain fog.

      If we know this to be an ordinary process in healthy brains, why is it hard to accept that brain fog simply starts at a lower level of mental exertion for those afflicted with this disease? Or that CBT’s value in this case is that it cuts a person’s baseline cognitive load?

      • Not this again

        A big part of the problem is that the CBT used in ME/CFS is different from CBT as most know it. As a coping strategy, geared to try to bring patients to a point where they understand and accept their limitations, and try to learn not to exceed them…who could have a problem with that? Well, it’s a problem in the sense that this is a front-line treatment, whereas when it’s used in other disabling conditions it’s at least secondary. But this is not the same as the CBT employed when, as stated in PACE,

        “CBT was done on the basis of the fear avoidance theory of chronic fatigue syndrome. This theory regards chronic fatigue syndrome as being reversible and that cognitive responses (fear of engaging in activity) and behavioural responses (avoidance of activity) are linked and interact with physiological processes to perpetuate fatigue. The aim of treatment was to change the behavioural and cognitive factors assumed to be responsible for perpetuation of the participant’s symptoms and disability. Therapeutic strategies guided participants to address unhelpful cognitions, including fears about symptoms or activity by testing them in behavioural experiments. These experiments consisted of establishing a baseline of activity and rest and a regular sleep pattern, and then making collaboratively planned gradual increases in both physical and mental activity. Furthermore, participants were helped to address social and emotional obstacles to improvement through problem-solving. Therapy manuals were based on manuals used in previous trials.”

        http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(11)60096-2/fulltext

        That material from those manuals, like what’s discussed here, discusses a therapy that this layman sees as being different from CBT as I understand it (having had it explained by numerous professionals, so it’s not like I’m completely unfamiliar,either).

        But no skeptics ever found this curious enough to take a look at. Which, again, is not a knock on Neuroskeptic. Though at this point (taking into consideration that my post was somewhat off-topic) I am starting to wonder what Neuroskeptic thinks about all this!

        • Christopher Lierle

          Sounds like you have strong knowledge of that study, and that’s the basis for your comments – which makes good sense, especially for a forum regarding data-driven skepticism.

          My (anecdotal) understanding is that CBT is rarely modified here in the US. Perhaps a trial comparing “normal CBT” to ME-specific CBT is in order? And it could include a combination of the two, as well?

          I agree with you and hope the Neuroskeptic takes a look at this.

  • OWilson

    Skepticism is just another form of reasoning, and is part of our biological survival gear.

    In the old days you might be offered a guaranteed rain dance in return for your chicken :)

    Skepticism is required when sorting out dire Doomsday predictions, like Hell, Holes in the ozone, Acid rain, AIDS, Doomesday Clock, Chernoble, Y2 millenium bug, fossil fuel depletion by 1985, Global Cooling, Killert Bees, Population Bomb, Chernobyl, China Syndrome, Killer Smog, Population 200 Million Environmental refugees by 2010, Arctic Ice free by 2015, Himalayan Glaciers gone by 2035, UFO’s, Sasquatch sightings, Loch Ness monsters, fairies, contact with the dead, Alien crop circles population bomb, fossil fuel depletion, Global cooling, the China syndrome, acid rain, killer smog, killer bees, AIDS, killer pollen, holes in the ozone, global cooling, end of times,

    Today it might be Global Warming, or even an amazing Health Plan that lets you keep your own plan and your own Doctor and saves you $2500 per year.

    Or some Presidential frontrunner telling you how she survived sniper fire, single handedly.

    Yep, maybe skepticism evolved for good reason ? :)

  • RMacCoun

    For what it’s worth, I discuss this issue on pp.274-275 of the attached paper, with some juicy examples of dilemmas posed by selective skepticism: http://socs.berkeley.edu/~maccoun/MacCoun_AnnualReview98.pdf

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      Interesting – thanks!

  • TheBlueStormtrooper

    I think one of the most important places for neuroskepticism is in the legal field. In line with your analysis, there is much excitement from mostly non-scientists in terms of what neuroscience breakthroughs mean for the application of the law. But there is much to be skeptical about too (see here: http://www.the-scientist.com/?articles.view/articleNo/44604/title/Opinion–Brain-Scans-in-the-Courtroom/), and this side needs a louder voice to prevent miscarriages of just.

  • Anonymouse

    It’s impossible to be a total skeptic, unless one doesn’t want to act or understand at all, because nothing can be taken for granted (or at least be met with more or less uncertainty), which would allow for it to be acted upon. So we can conclude that everyone is always a selective skeptic in the most general sense. Your point of course is about the implications of selective skepticism about things that aren’t common ground between different sites on a debate within a field which one claims to be skeptical about *as a whole* (like you do for neuro science).

    I don’t like idealism very much, if it doesn’t consider people’s psyche. So I don’t care for calling someone a bad or a good skeptic, when clearly people are biased in a way that doesn’t really allow them to question their own beliefs in the way they question those they don’t share themselves (and rightfully so*).
    So all that’s interesting to me about your point is the conclusion that from applying scrutiny to only one side follows implicit support for the other side. And while I’m inclined to agree with that (even though I remain skeptical about it (; ), I would still argue that partial criticism is better than none at all and that a balanced or exhaustive skepticism shouldn’t be desirable, let alone expected, for any individual, but for the (scientific) community as a whole. The latter, of course, is not reality either and, especially for academia, mechanisms that facilitate it would be very helpful indeed.

    * Because being a good skeptic in that sense would imply being very inefficient as an organism, similar to the clearly beneficial innate and learned biases of our visual system that can be exploited by visual illusions, which shouldn’t qualify us as “bad” at (veridical) vision.

  • s k

    ah, an article that touches nothing…

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      Well thanks for leaving a comment to match.

      • s k

        Perhaps I should have said I didn’t find the article helpful, but since others have…apologies. Neuro Skepticism may be better understood thru…hemispheric portrayals, as outlined in The Master and the Emissary, by McGilChrist. Some have found it very helpful. Not the usual trite stuff on brain lat. Skeptics have been around since alcohol….lol. Carvarka’s were the skeptics of the Vedas; again, a hemispheric depiction throughout the ages. Could it be so…if McGilChrist is correct about left-brain dominance? Yes. Skepticism is only the tiny interpreted aspect of our total nature. As Aldous said…external personal conciousness is reduced down to a mere trickle. Now, all of the good blogger skeptics can have their way…lol. One interesting thing about skeptic org’s is that they are often way-underinformed…or perhaps just ignoring evidence that doesn’t fit with skeptic design. Admittedly, the existential problem is a bit daunting….but not to worry.
        Bye.

        • Hugh Anderson

          “One interesting thing about skeptic org’s is that they are often way-underinformed”

          I’ve seen this a lot. Skeptics will always jump to defend the “scientific consensus” on any given subject without having the requisite deep understanding of that subject. On particular subjects they are the defenders of the status quo.

          Add to that the haughty tone skeptics tend to take and their overall intellectual hubris (looking at you Science Based Medicine), I would argue that many skeptics do more harm than good by alienating the very people they are pretending to try to educate. Most skeptics are really just talking to each other and not engaging in or learning from the wide body of scientific knowledge that would teach them how to influence human behavior and thought. If they were doing that, they wouldn’t be writing snarky self-righteous skeptic blogs and articles.

          Not talking about this particular blog as it were. :)

  • ddsouza

    skepticism is merely a stance that one has. it’s an important part of any liberal democracy. if you were accused of a murder you didn’t commit, you would hope that the jury would critically evaluate the evidence rather than trust the power of the tarot. you would hope that people critically evaluate the evidence when choosing which politician (if any) to vote for. and so on. it doesn’t tell you what you should believe or dismiss; it tells you how you should go about evaluating claims, etc.

  • Nathan Allan

    With intellectual prowess comes a growing pool of belief and opinion. We are doomed to wallow in our biases, especially confirmation bias, as our minds battle for us to minimize our cognitive dissonance. “We can’t… be… wrong!” our minds shout at us as we grapple with and cheery-pick as each new bit of information arrives. The degree to which we overcome, and ultimately find the greatest truth, is the degree to which we acknowledge our nature, embrace humility and and always proceed with the assumption that we’re wrong.

  • D Samuel Schwarzkopf

    Totally agree. And now you know why I invented the Devil’s Neuroscientist! 😉 (You already knew why I killed her off again… :P)

    Skepticism is a tricky thing. I am very skeptical towards claims of psi phenomena, climate change denial, creationism, homeopathy, or *ducks* social priming. I think the evidence and explanatory power of my alternative hypotheses to these things makes their claims implausible. But I do try to see things from both sides and I try never to treat any party as beyond reproach (but I am only human so I may fail at that).

    And while DNS has been exorcised and I am now a cautious advocate of Registered Reports, her skepticism about those kinds of things came from a real place. I don’t trust people with good ideas without any evidence that they truly make things better. The reason I support RRs is that 1. the theoretical arguments for them are sound, 2. I don’t believe they will bring down doom on scientific research, and 3. you can’t know if something is better or worse if you don’t try it.

  • Pingback: Why wouldn’t you share data? | NeuroNeurotic()

  • Pingback: Should We Defend the Scientific Consensus? - Neuroskeptic()

  • Pingback: Should We Defend the Scientific Consensus? – BlogON()

  • Pingback: Should we defend the scientific consensus? | |()

  • Pingback: Should we defend the scientific consensus? | NEWZE()

  • Pingback: Discover Magazine Rips Attacks on Vaxxed/Unvaxxed Study - Autism Investigated()

  • Pingback: Google()

  • Remi Gau

    I think you might find this old article of the Skeptical inquirer interesting: https://www.csicop.org/si/show/myth_of_consistent_skepticism_the_cautionary_case_of_albert_einstein

    • http://blogs.discovermagazine.com/neuroskeptic/ Neuroskeptic

      Thanks!

  • Pingback: The Selective Skepticism of Lynne McTaggart – KESIMPULAN()

  • Pingback: The Selective Skepticism of Lynne McTaggart – Gentle Bloom()

  • Pingback: The Selective Skepticism of Lynne McTaggart – Neuroskeptic | Very Top Secret Information()

  • Pingback: The Selective Skepticism of Lynne McTaggart - Neuroskeptic()

NEW ON DISCOVER
OPEN
CITIZEN SCIENCE
ADVERTISEMENT

Neuroskeptic

No brain. No gain.

About Neuroskeptic

Neuroskeptic is a British neuroscientist who takes a skeptical look at his own field, and beyond. His blog offers a look at the latest developments in neuroscience, psychiatry and psychology through a critical lens.

ADVERTISEMENT

See More

@Neuro_Skeptic on Twitter

ADVERTISEMENT

Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Collapse bottom bar
+