I think the population should be more useful and impactful. Even worse is that on a colony worlds, population doesn't increase output of those worlds but eats more and more food, so having more than 1 pop is a disadvantage despite the world's size. Considering citizen's specialization on a core worlds, i think each specialization should increase fixed value instead of a percentage like 0.5 to manufacturing or research and so on and farmers 1.5 food for example (considering that they eat 1), but amount of farmers should be limited by the planet fertility and manufacturers should increase the pollution. And i think there should be an interface to move your citizens betveen the core colony and assigned colony worlds, maybe 100 income per pop transfer.
P.S. I think i found a bug, so when pirates bombed my colony world and captured it, i captured it back and it was unassigned from my core world and now i can't assign it back. It already has 2 pop and it only eats my food without providing anything