Neandertal hybridization & Haldane's rule

By Razib Khan | April 27, 2011 11:35 pm

Mr. James Winters at A Replicated Typo pointed me to a short hypothesis paper, Neanderthal-human Hybrids. This paper argues that selective mating of Neandertal males with females of human populations which had left Africa more recently, combined with Haldane’s rule, explains three facts:

- The lack of Neandertal Y chromosomal lineages in modern humans.

- The lack of Neandertal mtDNA lineages in modern humans.

- The probable existence of Neandertal autosomal ancestry in modern humans.

If you don’t know, Haldane’s rule basically suggests that there’s going to be some sort of breakdown in the heterogametic sex. In humans females are homogametic, XX, and males are heterogametic, XY. The breakdown need not be death (or spontaneous abortion). It could be sterility (e.g., some mutation or genetic incompatibility which results in the malfunctioning of the flagella of sperm would do it).

So you have a scenario where only Neandertal males are interbreeding with the intrusive groups from the south. The hybrids from these pairings would then lack Neandertal mtDNA, since mtDNA is passed only from mothers. But the male offspring would have Neandertal Y chromosomes. This is where Haldane’s rule kicks in: these males in their turn would not reproduce. Therefore only the female hybrids would pass on their genes. These females obviously don’t pass on a Y chromosome. And, they would pass on their non-Neandertal mother’s mtDNA.

Obviously this makes logical sense. How plausible do I judge it? That depends on the other options and the probabilities in the moving parts of the model above. My main issue with the idea of Haldane’s rule being operative in Neandertal-non-Neandertal pairings is this: the two lineages had not been separated for very long at all. The authors give ~250,000 years for the most recent common ancestor. Let’s just double that. That still isn’t that big of a divergence. A few years ago I read some stuff on hybridization in mammals. There’s some pretty straightforward reasons having to do with gestation why this is more of an issue in our lineage than birds, for example, where you have instances of viable crosses between species whose last common ancestor lived tens of millions of years in the past. But that doesn’t speak to the issue of Haldane’s rule necessarily. The problems with interfertility tend to crop up on the order of millions of years, not hundreds of thousands.

In any case, what about the alternatives? There could have been some sort of selective bias against mtDNA and Y chromosomal lineages. This can be straightforward biological. Imagine that Neandertal mtDNA is correlated with some diseases with reduce fitness. The authors allude to this sort of issue. But it might be social. Across Latin America there has been wholesale replacement of Amerindian Y chromosomal lineages among mixed-race populations. In fact you have replications across many societies of European Y chromosomal lineages + non-European mtDNA lineages being dominant, with variation in autosomes (e.g., in Mexico the autosome is balanced, in Argentina it is mostly European). There is also the issue that mtDNA and Y chromosomal lineages are subject to more vigorous stochastic dynamics because of smaller effective population sizes than autosomes. Autosomes are a combination of both parental contributions, but the uniparental lineages are passed from only one. Males are a total dead end in regards to the propagation of mtDNA lineages since they do not pass them on, while females naturally do not have a Y chromosome. The Neandertal mtDNA and Y lineages may simply have gone extinct, which is more probable if they were a small minority in the human population ~30,000 years before the present (the probability that a lineage with “fix” and replace all others is proportional to its frequency at time t = 0).

But really the main issue here for me really is the plausibility of hybrid incompatibility between Neandertals and non-Neandertals. This was a common idea a few years ago before the evidence for Neandertal-non-Neandertal admixture, and I’d started to get skeptical of it based on comparisons to other mammals. But now we have more thorough genetic data. To the left is a table from the supplement of Genetic history of an archaic hominin group from Denisova Cave in Siberia. It is showing the time since the last common ancestors between pairs of populations (F = French, the rest of the rows are the same as the columns). I wouldn’t take the dates that seriously. What I want to point out is that the last common ancestor between Neandertals and other human populations isn’t even a multiplicative factor greater than that between Africans and non-Africans. These particular estimates might be wrong in the details of their magnitude, but I think before we assent to the probability of hybrid incompatibilities we need to consider the high likelihood that Neandertals just weren’t nearly as different as we might think, or have thought.

The following video is for entertainment purposes only:

CATEGORIZED UNDER: Evolution, Genetics, Genomics
  • Meng Bomin

    Consider me entertained.

  • toto

    I liked the slit pupils!

  • Justin Giancola

    gen-Ome! geez dialects!

    One thing I think he’s on the right track on is Neanderthals looked like bad-asses! ;)

  • Peter Ellis

    As I’ve suggested in a comment here before, I suspect Haldane’s rule is a plausible contributor to the patterns of gene flow between Neandertal and non-Neandertal human populations. I’m less convinced by the paper’s attempt to link the Y and mitochondrial data though – that makes no sense. Haldane’s rule does not pose any barriers to mitochondrial gene flow.

    Although 250,000 years is indeed comparatively short, it’s a similar ballpark to the divergence between Mus musculus subspecies (500,000 yr), and these clearly show hybridisation effects. Depending on exactly which populations you look at, hybrids between domesticus and musculus subspecies can either show full fertility, male sterility, or even non-reciprocal male sterility depending on which subspecies contributes the X / Y chromosomes. If you look at hybrid zones between musculus and domesticus populations, there is evidence for sex ratio skewing depending on which sex chromosomes are present on which genetic background, again supporting the contention that sexually antagonistic genes are contributing to speciation.

    It’s worth pointing out that it needn’t be complete hybrid sterility: hybrid breakdown over the course of a few generations would also form an effective barrier to gene flow. This is what you see in crosses of molossinus and laboratory mouse (predominantly domesticus). In the F1 generation, both males and females are fertile, but male fertility is lost as you back-cross the molossinus X onto the domesticus background. There’s reasonable transcriptional evidence for a genomic conflict between X and Y-linked genes, with interacting autosomal modifiers.

    Obviously there are differences between rodent and human: generation time and metabolic rate, for a start. 500,000 years is also substantially longer than the interval between the Neandertal / non-Neandertal split and the subsequent gene flow. On the other hand, we don’t know when the diverging mouse populations started to show hybrid breakdown / sterility, all we know is that it took less than 500,000 years.

  • jb

    I’ve always felt that reconstructions of Neanderthals generally looked far too much like modern Europeans. We don’t need to go all the way to slit-eyed killer apes (!), but the fact that the genetic distance between Neanderthals and modern Europeans is greater than the distance between modern Europeans and Africans suggests that the difference in appearance was probably also greater. I wonder what they really looked like! (I do think it’s at least possible they had fur!!!)

  • dave chamberlin

    You just know some producers over at the Discover Channel are going to run with Neanderthals, Vicious Ape Rapers from the North. They have exhausted UFO’s, Nostradamus, Hogs Gone Wild, Ghosts and Big foot. Poor little timid humans, all they did was go looking for new food sources and what happens when they are all snuggled up around the night fire, ape rape attack.

  • miko

    “…but the fact that the genetic distance between Neanderthals and modern Europeans is greater than the distance between modern Europeans and Africans suggests that the difference in appearance was probably also greater.”

    It doesn’t. The phenotypes that make up gestalt “appearance” are highly labile. One could argue (I’m not, necessarily) that adaptation for living in Europe (which apparently includes bathing aversion and love of fluorescent clothing) led to broadly convergent changes in facial features in Neandertals and European Hs. Selection on many traits related to physiology are likely to secondarily affect appearance due to the shared developmental origin of craniofacial, endocrine and enteric nervous system tissues in the neural crest.

    That’s all hand-waving, but I don’t think genetic distance is necessarily a good proxy for what we would experience as facial samey-ness, particularly between populations, whereas the selective environment might be under-appreciated as a systematic source of variation in facial features.

    That said, I don’t really think any of those reconstructions look much like Europeans, except for the one that looks like Bill Maher.

    And Danny Vendramini is my new Erich von Daniken. What a nut! I wonder if it’s optioned yet.

  • pheldespat

    I’ve got one doubt only: What’s a “comment ancestor”? :)

  • RafeK

    Very Entertaining indeed.

    To back up Miko’s point about degree of genetica relatedness and similarity of appearance, Papaun tribesmen are more closely related to Han Chinese then West Africans were appearances would indicate quite the opposite.

  • Razib Khan

    tx! funny.

  • ohwilleke

    I’ve been beating this drum for months now. I’m glad to see that someone else is starting to accept it as well.

    Also, there is a pretty easy way to solve the question of why all Neanderthal admixture in modern humans comes from Neanderthals males, which is the missing link in the analysis above. All you need is a scenario in which Neanderthal-human relationships are in the nature of rape/flings/seasonal meetups rather than being stable long term marriages, or (probably less plausibly) with Neanderthal-human relationships being matrilocal (which seems to be at odds with the very little that we know about Neanderthal social structure). Either way, the hybrid kids end up with their mother’s tribes.

    In the case of modern human mothers (who were pregnant from Neanderthal fathers) those hybrid kids join the modern human gene pool. In the case of Neanderthal mothers (who were pregnant from modern human fathers) those hybrids end up in the Neanderthal gene pool.

    The trouble is that the Neanderthal gene pool is a pond that goes dry when the Neanderthals go extinct, so that mixed species kids with Neanderthal mothers go extinct along with their Neanderthal maternal extended families, leaving only the hybrid kids of modern human mothers in the modern gene pool.

  • Ian

    At first glance, the Amerindian example seems questionable, since that pattern reflects not only contact, but the power differential between coloniser and colonised. But OHWilleke’s comment (#11) got me thinking – just because humans eventually displaced Neanderthals doesn’t mean that they got the upper hand in encounters between the two groups.

    What if Neanderthals had the upper hand in these interactions? Hunter-gatherers can’t steal material possessions, but they can steal women. If you’re talking about a prolonged tributary relationship, it’s also conceivable that the Neanderthals were able to recognise (and claim) hybrid sons. (It wouldn’t be such a stretch to imagine Neanderthal society prizing sons more than daughters – it would fit well within what we find in many modern human societies.)

    I’m thinking of the Island Caribs as a (partial) model. Men and women spoke different languages – the men reportedly spoke the Carib language, while the women spoke an Arawakan language (presumably reflecting the long-term import of captive Taino women).

    There’s a weakness to such a model – if most hybrid sons were recovered by their fathers, then the contact would have to be prolonged. If there was prolonged contact, then the human-to-Neanderthal introgression would probably be more pronounced than in Neanderthal-to-human. The paucity of putative hybrid fossils might argue against this…though, on the other hand, despite the lack of fossils, we know that hybrids existed.

  • Torbjörn Larsson, OM

    FWIW, John Hawks, who should be an expert at this, said this a year ago:

    “I keep seeing people, who really ought to know better, saying that the new Neandertal genome results show that the gene flow must have been Neandertal men mating with modern human women, and not the other way around.

    You see, they’re fixated on the idea that the mtDNA showed no signs that the Neandertal clade survived into the present-day population. That result really convinced some people that interbreeding was impossible. They’re flummoxed that some of the rest of the genome has significant signs of intermixture. It’s like their world is spinning out of control. I’m not naming any names, but if you’ve followed much of the press around the Neandertal genome, you’ve probably seen this suggestion.

    I don’t know why it hasn’t occurred to them that the Neandertal mtDNA type was probably lost because of natural selection.”

    I like a more balanced approach though, such as in the post as I read it.

    @ dave chamberlain:

    Don’t you think Vendramini, a movie producer/writer/director, haven’t thought about that? From the beginning, perhaps.

    [Ouch! He turned up in the ads too.]

  • Eze

    When DNA testing becomes commonplace in another 5-15 years, I envision that a Neanderthal Y or MT-DNA lineage will be found in a modern Hs. at random.

  • ohwilleke

    “I don’t know why it hasn’t occurred to them that the Neandertal mtDNA type was probably lost because of natural selection.”

    While this is possible, there doesn’t seem to be particularly strong evidence of natural selection playing a strong role in modern human mtDNA variation. The mtDNA variation looks like is a better fit for selective neutrality with it coming along for the ride of trends driven by other genetic and/or cultural drivers of natural selection that happen to coincide with mtDNA variants.

    Hawks recent findings that the Neanderthal DNA found in East Eurasians is mostly different than the Neanderthal DNA found in West Eurasians also argues for the case that most of our Neanderthal DNA legacy, while not decisively counterproductive from a selective perspective, also didn’t provide decisive selective advantages. The case for selective advantage can only be made if the number of genes found in both West and East Eurasian Neanderthal DNA legacies are a larger share of the total than one would expect from random chance.

    The selective factors related to mtDNA would probably relate to metabolic rate, as mitchondria are intimately involved in regulating metabolism. This is a trait that one would expect Neanderthals in colder climes to have better adapted to for colder climates than modern humans who would be more recent African expatriots where it is warmer, at least for those modern humans migrating to colder areas as many were doing in the era of Neanderthal-modern human interaction. Thus, it would be quite surprising that modern human mtDNA would have selective advantage relative to Neanderthal DNA.

    I also don’t think that Eze @14 is correct that a random DNA test has much of a chance of discovering a Neanderthal Y or mtDNA lineage in a modern human in the next 5-15 years. The aggregate sample size of all Eurasians who have had uniparental DNA haplogroup typing already is on the order of tens of thousands if not the hundreds of thousands, with atypical relict populations mostly oversampled, with not a single such case identified so far.

    Given the amount of population expansion (something on the order of ten thousand fold) that the world has seen in the last 30,000 years (the youngest possible origin for Nenaderthal DNA), it is very hard to imagine that a gene with a thousand plus generations of age in a period of profound population expansion would have stayed in the population, but be so rare that it or a phylogenetic descendant of it, wouldn’t have shown up in a sample as large as the one that already exists.

  • Matt B.

    Um, does Vendramini know that gorillas aren’t bipedal? I don’t think chimps are really all that bipedal. He says neanderthals are primates, but doesn’t note that they’re more specifically hominids. And it’s interesting how his voice echoes the same when he’s ostensibly outdoors as when he’s indoors.

  • Pingback: I’m part Neanderthal, and proud of it! - Genomes Are Us

  • Walter Sobchak

    “selective mating of Neanderthal males with females of human populations which had left Africa more recently”

    Can’t be. The Africans must have been the aggressors, and they got the women. Otherwise we would be Neanderthals wondering what happened to the Africans.

  • Patrick Buck

    This adds a whole new meaning to one of my late father’s favorite expressions to describe bratty kids or otherwise out-of-control people “as wild as a raped ape.” Seriously, though, I agree with the advisory under the video “For entertainment purposes only.” I was, indeed, entertained.

  • Day Brown

    Talk to a midwife. I’ve been quoting for years online now one who looked at the robust Neanderthal pelvis- which does not crack wider during birthing- saying they would not have been able to safely deliver a hybrid, but a Cro Magnon woman would.

  • Christopher Rees

    The consumption of homo sapiens flesh, particularly the brain, probably gives a clue to the eventual extinction of homo sapiens neanderthalensis. Look to a prion condition like kuru in the central New Guinea highlands that, in the late 1940′s and through the 1950′s, nearly exterminated the tribes that had lived there for tens of thousands of years. Our ancestors did not hunt the neanderthals to extinction. Our brain prions killed them off. Kind of a poetic justice, that!

  • Russell W

    @# 3 Justin Giancola,

    “geeze dialects!” . Don’t you speak with a dialect?

    The video is definitely for entertainment only.

  • Øyvind Kvernvold Myhre

    Please do the statistics, folks. mtDNA lines disappear – not because of natural selections, and not because the people carrying the lines didn’t reproduce, but because of random statistical effects. A personal example: My paternal grandmother has approximately fifty living descendents, all carrying fractions of her DNA. However, her mtDNA line is gone forever. That’s because she had five sons and two daughters. Those two daughters didn’t have any daughters, just sons! On the other hand, my maternal grandmother has only ten living descendents. She had one daughter, who had one daughter, who had two daughters. Her mtDNA line lives on!

    In a stable population, one where each reproducing female leaves exactly two reproducing children, 90 % of the original mtDNA lines will be broken in just 34 generations. That’s less than 1.000 years. So if you start with 1.000 female Neanderthals in a mixed population, less than 100 Neanderthal mtDNA lines will survive after 1.000 years. After another 1.000 years, less than 10. After yet another 1.000 years… You can do the same type of calculation regarding Neanderthal Y chromosome lines. All taken together: If Neanderthals were a small minority of our common ancestors, it is extremely unlikely that any two Neanderthal genes would survive after 30.000 years. (In effect, mtDNA and the Y chromosome are inherited as just one gene, since they’re inherited in one piece.) Conclusion: We don’t have to construct intricate theories of marriage customs or patterns of reproduction in order to explain the lack of Neanderthal mtDNA or Neanderthal Y chromosomes today; we only have to do some reasonable math.

    For thirty years, the ruling orthodoxy – the ‘Recent out of Africa’ school – has been leaning heavily on just one gene, out of our 25.000. In reality, each gene has its own tree of ancestry! The total picture will be incredibly complex, and we’ve just started to put together a few pieces of the jigsaw. What we do know is that the average person carries approximately 1.000 genes inherited from our Neanderthal ancestors. (My own ‘Neanderthal bun’ at the back of my skull confirms my mixed ancestry!) Most of the rest of our genes in all likelyhood are of recent African origin (‘recent’ meaning 100.000 years or less), but we don’t KNOW this yet. We’ll know much more once we start seeing DNA from Asian Homo Erectus, and I’m convinced we’re in for a lot of surprises. There’s a revolution going on in the way we perceive who we are and where we come from.

  • John Uri

    Entertainment, yes. Science, no.

  • Razib Khan

    Please do the statistics, folks. mtDNA lines disappear

    who you are talking to? you notice i mentioned this in my post right, since you read it?

  • Craig

    At 4:13 he holds the skull so it’s looking at the ceiling and says it’s got a protruding face. Not if you hold it properly so the eyes are looking forward.

  • Øyvind Kvernvold Myhre

    To Razib Khan: You’re quite right, mate. You did point it out! However, lots of people – including several famous paleoanthropologists – have disregarded this simple fact for thirty years! As more fossils are analyzed, we should expect to see lots of mtDNA lines that are lost today. These lines will be found not only in Neandertals and Denisovans, but also in ‘modern’ Homo Sapiens, in Africa as well as elsewhere. By no means does this mean that those people have no living descendents!

    Several years ago, an Australian team published their results from research on fossils from Lake Mungo – perfectly ‘modern’ specimens, I should add. The mtDNA they found fell outside the accepted tree rooted in the famous ‘African Eve’, and as a result, their research was largely disregarded. Whether or not their results were caused by contamination, as some critics claimed, such a result is exactly what one should expect – on purely statistical grounds.

  • Onur

    A lot of Vendramini’s arguments seem to be pure speculation without solid basis. But one thing struck my mind: Do the hard tissue based parts (e.g., position and size of the eyes) of Vendramini and Balseiro’s Neanderthal reconstruction have any scientific validity? I am asking this as, unlike genetics, I am pretty uninformed about facial reconstruction.


Discover's Newsletter

Sign up to get the latest science news delivered weekly right to your inbox!

Gene Expression

This blog is about evolution, genetics, genomics and their interstices. Please beware that comments are aggressively moderated. Uncivil or churlish comments will likely get you banned immediately, so make any contribution count!

About Razib Khan

I have degrees in biology and biochemistry, a passion for genetics, history, and philosophy, and shrimp is my favorite food. In relation to nationality I'm a American Northwesterner, in politics I'm a reactionary, and as for religion I have none (I'm an atheist). If you want to know more, see the links at


See More


RSS Razib’s Pinboard

Edifying books

Collapse bottom bar

Login to your Account

E-mail address:
Remember me
Forgot your password?
No problem. Click here to have it e-mailed to you.

Not Registered Yet?

Register now for FREE. Registration only takes a few minutes to complete. Register now »