If you delve into the wildest depths of the scientific literature, you will find a trilogy of papers so weird, that they have become legendary.
In these articles, spanning a 12 year period, author Jarl Flensmark says that heeled shoes cause mental illness, while flat footwear promotes brain health:
A dubious paper just published in Molecular Neurobiology makes the suggestion that all military recruits should be offered genetic testing to assess their risk of PTSD. According to the authors, Kenneth Blum et al.,
We hypothesize that, even before combat, soldiers with a childhood background of violence (or with a familial susceptibility risk) would benefit from being genotyped for high-risk alleles (DNA variants). This process may assist us in identifying potential military candidates who would be less well suited for combat than those without high-risk alleles.
Fortunately, the authors claim, such a test already exists, and it’s called the Genetic Addiction Risk Score (GARS).Read More
A paper in a peer-reviewed medical journal makes the suggestion that physicist Stephen Hawking’s disability, which famously confined him to a wheelchair and robbed him of his speech, was psychosomatic in nature.
Hmm. I think this says more about the author than it does about Hawking.Read More
A paper in PNAS got some attention on Twitter recently. It’s called Childhood trauma history is linked to abnormal brain connectivity in major depression and in it, the authors Yu et al. report finding (as per the Significance Statement)
A dramatic primary association of brain resting-state network (RSN) connectivity abnormalities with a history of childhood trauma in major depressive disorder (MDD).
The authors go on to note that even though “the brain imaging took place decades after trauma occurrence, the scar of prior trauma was evident in functional dysconnectivity.”
Now, I think that this talk of dramatic scarring is overblown, but in this case there’s also a wider issue with the use of a statistical method which easily lends itself to misleading interpretations – canonical correlation analysis (CCA).
First, we’ll look at what Yu et al. did. In a sample of 189 unmedicated patients with depression, Yu et al. measured the resting-state functional connectivity of the brain using fMRI. They then analyzed this to give a total of 55 connection strengths for each individual. Each of these 55 measures reflects the functional coupling between two brain networks.
For each patient, Yu et al. also administered questionnaires measuring personality, depression and anxiety symptoms, and history of trauma. These measures were then compressed into 4 clinical clusters, (i) anxious misery (ii) positive traits (iii) physical and emotional neglect or abuse, and (iv) sexual abuse.
This is where the CCA comes in. CCA is a method for extracting statistical associations between two sets of variables. Here one set was the 55 brain connectivity measures, and the other was the 4 clinical clusters. Yu et al.’s CCA revealed a single, strong association (or ‘mode of variation’) between the two variable sets:
But the result isn’t as impressive as it seems, because it’s a CCA result. CCA is guaranteed to find the best possible correlation between two sets of variables, essentially by combining the variables (via a weighted sum) in whatever way maximizes the correlation coefficient. In other words, it is guaranteed to over-fit and over-estimate the association.
Yu et al. show this, as they found that using a permutation procedure (which eliminates any true associations) the CCA still produced a mean correlation coefficient of r=0.55. In 5% of cases, the CCA was lucky enough to hit r=0.62 or higher. Remember that the ‘true’ correlation is zero in this case! CCA is able to magic up a strong correlation of 0.55 or higher from out of thin air.
The observed correlation of r=0.68 is statistically significant, because it’s higher than the 95% null of 0.62, but it’s not much higher. In other words, while there does seem to be some true relationship between the brain and behavior variables here, it is almost certainly much weaker than it appears.
(Yu et al. in their paper also carried out a comparison of depressed patients to healthy controls, which does not rely on CCA, and which I’m not discussing here.)
So what is the use of CCA, if it is guaranteed to overfit the data? Well, it can be useful so long as you have two (or more) independent datasets, allowing you to test the validity of the CCA model, derived from one dataset, in another. The CCA would be overfitted to the first dataset, but by testing it in the second dataset, we can know how much of the correlation is real.
Unfortunately, Yu et al. is not the only paper to adopt a single-sample CCA approach. A well-cited paper Smith et al. (2015) in Nature Neuroscience, which Yu et al. refer to several times, did the same thing. (I blogged about it at the time, rather un-skeptically).
Smith et al. compared brain functional connectivity to behaviour and lifestyle variables, and found a mode of CCA variation with a spectacularly strong correlation of r=0.8723. But the 95% significance threshold under the permuted null hypothesis turned out to be an almost-as-spectacular r=0.84! So, just as with Yu et al., the observed result was significant, but only slightly better than CCA produced by chance alone.
In fact, Smith et al. went on to test the validity of the CCA by running CCA for 80% of the dataset (‘training set’) and testing it in the remaining left-out 20%. This is a kind of rough-and-ready approximation of using a second dataset. Smith et al. found that the correlation in the left-out data was r=0.25 – a much more modest result, although still something.
I would say that this kind of train/test analysis should be a bare minimum in any neuroscience CCA paper. I suspect that if it were applied in Yu et al.’s case the correlation would be small.
A Swedish company called Emotra make a device to detect someone’s risk of suicide based on measuring the body’s autonomic responses to certain sounds. It’s called EDOR®.
I’ve been blogging about this machine for the past 18 months (1, 2, 3) because such a product, if it worked, would be very important. It could help save countless lives. Unfortunately, I don’t think EDOR® has been proven to be effective. As I’ve argued in my previous posts, the evidence just isn’t there yet.
Suppose, if you will, that alien scientists came down to Earth and began to study the local lifeforms. But let’s suppose that these aliens arrive by the side of a busy expressway, and stay there. Our extraterrestrials might conclude that cars are the dominant inhabitants of Earth.
Cars clearly exhibit intelligent behaviour, being able to navigate around obstacles and follow complex instructions on road signs. How, the aliens may wonder, do the cars manage this? What is the seat of car intelligence?
After some experimentation, the aliens would eventually work out that it is a carbon-based organ inside the car – aka the driver – that is controlling the vehicle. The driver is the one making the decisions, and the rest of the car is just following its commands. The proof of this is that if the aliens remove a car’s driver, it stops moving.
Experiments could be conducted to find out the function of different parts of the driver. Lesion the driver’s feet, for instance, and the car would have trouble braking and accelerating, while arm damage would produce selective deficits in turning.
These lesion experiments could be supplemented with imaging studies, in which aliens scan the car and record the activity in the different parts of the driver. Just as lesion studies predicted, the feet would be more active when braking, while arm movements would predict turns.
Eventually, “driverscience” – the study of the driver and its function – would become a well-developed field of alien research. Everyalien would know that the arms are responsible for steering while the feet control speed, and so on.
I hope it’s obvious that in this scenario I’m drawing an analogy between the driver, as controller of the car, and the brain as the controller of the body. The aliens have concluded that the human driver is the “brain” of the car.
Knowing neuroscience as we do, we might be tempted to say that the aliens are mistaken, and that only the driver’s brain is actually responsible for controlling the car. The aliens are wrong in thinking that hands or feet or other body-parts have anything to do with intelligence, we think. The brain is the driver of the driver.
Yet who is to say that we are not falling into the same trap as the aliens, when we attribute intelligence to particular brain regions? If we talk about the amygdala (say) “recognising a threat”, we imply that this region, by itself, is carrying out an intelligent function; but this might be as mistaken as to say that the feet “recognize” that the car is going too fast.
Scientific papers should have two Discussion sections – one written by the authors, and the other by an independent researcher.
According to a new paper from Michael S. Avidan, John P. A. Ioannidis and George A. Mashour, this “second discussant” system could help ensure more balanced and objective inference in science.
I just came across a paper with an interesting title: The Mutant Says in His Heart, “There Is No God”.
The conclusions of this work are even more interesting. According to the authors, Edward Dutton et al., humans evolved to be religious and atheism is caused (in part) by mutational damage to our normal, religious DNA. Atheists, in other words, are genetic degenerates.
“Foreign Accent Syndrome” (FAS) is a rare disorder in which patients start to speak with a foreign or regional tone. This striking condition is often associated with brain damage, such as stroke. Presumably, the lesion affects the neural pathways by which the brain controls the tongue and vocal cords, thus producing a strange sounding speech.
Yet there may be more to FAS than meets the eye (or ear). According to a new paper in the Journal of Neurology, Neurosurgery and Psychiatry, many or even most cases of FAS are ‘functional’, meaning that the cause of the symptoms lies in psychological processes rather than a brain lesion.