The Neanderthals disappeared about 30,000 years ago, but little pieces of them live on in the form of DNA sequences scattered through the modern human genome. A new study by geneticists at the University of California, Davis, shows why these traces of our closest relatives are slowly being removed by natural selection.

“On average, there has been weak but widespread selection against Neanderthal genes,” said Graham Coop, professor in the UC Davis Department of Evolution and Ecology and Center for Population Biology, and senior author on a paper describing the work published Nov. 8 in the journal PLOS Genetics. That selection seems to be a consequence of a small population of Neanderthals mixing with a much larger population of modern humans.

Neanderthals split from our African ancestors over half a million years ago, and lived in Europe and Central Asia until a few tens of thousands of years ago. Archaeological discoveries have shown that they had quite a sophisticated culture, Coop said. Thanks to DNA samples retrieved from a number of fossils, we have enough data on the Neanderthal genome to identify their genes among ours.

When modern humans left Africa about 50,000 to 80,000 years ago and spread through Europe and Asia, they interbred with Neanderthals. The first hybrid offspring would have been, on average, a 50-50 mix of modern human and Neanderthal genes, and could then have themselves bred with modern humans, Neanderthals or other hybrids.

So what happened to the Neanderthal DNA? Today, Neanderthal genes are a few percent of the genome of people of European ancestry, a little more common in people of East Asian descent, and almost absent in people of African ancestry.

Coop and postdoctoral researchers Ivan Juric and Simon Aeschbacher devised methods to measure the degree of natural selection acting on Neanderthal DNA in the human genome.

One hypothesis has been that Neanderthals quickly became genetically incompatible with modern humans, so their hybrid offspring were not “fit” in evolutionary terms – they either failed to thrive or were not fertile.

Weak but widespread selection against Neanderthal genes

The researchers found something different. Rather than showing strong selection against a few Neanderthal genes, they found weak, but widespread selection against many Neanderthal DNA sequences that is slowly removing them from our genome.

Coop said that’s consistent with a small, isolated population of Neanderthals mixing with a much larger population of modern humans. Inbreeding in small populations means that genetic variants can remain common even if they’re harmful to some degree. But when they mix into a larger population, natural selection starts to act against those variants and weed them out.

“The human population size has historically been much larger, and this is important since selection is more efficient at removing deleterious variants in large populations,” Juric said. “Weakly deleterious variants that could persist in Neanderthals could not persist in humans. We think that this simple explanation can account for the pattern of Neanderthal ancestry that we see today along the genome of modern humans.”

The findings are consistent with other recently published work. If Neanderthals had been more numerous when modern humans encountered them, we might have a different mix of Neanderthal and human genes, Juric said.

The work was supported by grants from the National Science Foundation, the National Institutes of Health and the Swiss National Science Foundation. Juric is currently at 23andMe Inc., Mountain View, California.

More: Listen to Graham Coop talk about this work on our podcast, Three Minute Egghead.