It's all about genetic drift and there is well documented evidence that it's a highly recurring phenomenon. When the neanderthals got pocketed into the cold wastelands of Europe and the humanoid species that followed them entered Europe they both had no choice but to interbreed in order to survive. By that time the neanderthals were already selectively few and those that survived were going to seem least "mixed". Fast forward millenniums into the future in China. In populating China's north it was relatively more mixed in the beginning just like neanderthals were. It was also a place where far northerners/northerners westerners southerners easterners collide and in these areas people are bound to mix. Their activity creates civilization. But after all is said and done mass migrations for whatever reason create diaspora and the genetic drift occurs again. Northerners went in a southern direction and westerners in a eastern direction to populate and or sinicize and what happened? The north and west drifted because those that were left in those areas were like those pocketed neanderthals that found ways to survive that others found difficult and gave up. The only reason why those appear least mixed is because of the drift that occurs which puts a lead on what can and has survived while diminishing or putting a cap on those numbers that end up migrating elsewhere because they found it difficult to survive.
I think you're speaking from ruling class pov, yes Chinese dynasties over long period of centuries did dominated by Northerners in origin but majority of Han Chinese population were from Yangtze river areas. While atDNA of Han Chinese show common branch, yDNA of Han Chinese shows irregular patterns showing there's mixture. Cultural influence is still much Yellow river culture but language influence is very much Yangtze river. This is why "language" have become main identifier of origin for ancient people because during ancient period, early humans rarely adopt foreign spoken language because there was no schooling, it was mothers who taught their kids. But after nation was formed and education and clanship became important, more & more people learned from others.