We think of DNA as the vitally important molecules that carry genetic instructions for most living things, including ourselves. But not all DNA actually codes proteins; now, we're finding more and more functions involving the non-coding DNA scientists used to think of as 'junk'.

A new study suggests that satellite DNA – a type of non-coding DNA arranged in long, repetitive, apparently nonsensical strings of genetic material – may be the reason why different species can't successfully breed with each other.

It appears that satellite DNA plays an essential role in keeping all of a cell's individual chromosomes together in a single nucleus, through the work of cellular proteins.

According to biologists Madhav Jagannathan and Yukiko Yamashita who authored the new study, that important role is managed differently in each species, leading to genetic incompatibility. The clash of the different strategies between species may be what causes chromosomes to scatter outside of the nucleus, at least in part, preventing reproduction.

"We propose a unifying framework that explains how the widely observed satellite DNA divergence between closely related species can cause reproductive isolation," they write in their paper.

This "satellite DNA divergence" has been well established in previous research, leading to suspicions about its role in speciation. In the case of the chimpanzee genome and the human genome, for example, the protein-coding DNA is almost identical, while the 'junk' DNA is almost entirely different.

In this new study, experimenting on the fruit fly Drosophila melanogaster, the researchers noticed that deleting a gene which produces a protein called Prod – which binds to a specific bit of satellite DNA – caused the flies to die, as their chromosomes scattered outside the cell nucleus. However, that crucial bit of satellite DNA is missing altogether in the flies' nearest relatives, which survive just fine without it.

That suggests these important non-coding sequences of DNA material have evolved differently between species. To take a closer look, the team examined hybrid offspring bred from a D. melanogaster female and a male from the closely related D. simulans species.

Flies bred in this way usually die very quickly or end up sterile. In this case, an examination of the tissue of the hybrid offspring confirmed what the researchers had suspected – that the chromosomes (the DNA packages necessary for reproduction) were being disrupted here as well.

"When we looked at those hybrid tissues, it was very clear that their phenotype was exactly the same as if you had disrupted the satellite DNA-​mediated chromosomal organization of a pure species," says Yamashita, who works at the Massachusetts Institute of Technology (MIT).

"The chromosomes were scattered, and not encapsulated in a single nucleus."

Digging even further, the researchers produced healthy hybrid flies by removing the genes known to damage hybrid offspring (called 'hybrid incompatibility genes') from their parent flies. These incompatibility genes are known to localize to satellite DNA in the pure species.

Satellite DNA mutates fairly regularly, and the researchers think that the proteins that bind to satellite DNA to keep chromosomes together have to evolve to keep up. This then gives each species its own different strategy when it comes to satellite DNA operations.

Next, the team wants to try and design a protein that successfully binds across the satellite DNA of two species, keeping the chromosomes where they should be. That could enable viable offspring between those species, but it will take years to realize.

"Our study lays the foundation for understanding hybrid incompatibility at a cellular level in Drosophila as well as other eukaryotes," write the researchers.

The research has been published in Molecular Biology and Evolution.