Build a colony (or a construction) ship at the Core world you want it to come from, and use that population unit to load on that ship when it is built.
Then move that colony (or construction) ship to the world where you want the population unit to go, i.e. LAND the ship on the planet to make sure it goes to the right Core World.
Then DISBAND that Colony (or construction) ship, i.e. left click twice on the ship and then when the detailed screen for information on that ship comes up, you get at the bottom of the screen 2 options: rename the ship or Disband the ship.
After that the ship no longer exists and can not be used again.
If you do Not Disband the ship, the population unit will remain attached to that colony (or construction) ship, and will not join that Core World.
So it is going to cost you the price of of building that ship (or using a Draft Colonist Executive order using Control Points).
PS I tried designing and building a Passenger Ship by using both a Colony and a Construction Module on the same ship. But the operational test of the Passenger Ship design failed, since the population points remained linked to the Passenger Ship and never got off it (unless I Disbanded the Ship), even after it landed on a Core World.
So it is expensive but possible to move a population point to another Core World.
added: You should really only need to do this in cases like the target Core World only has Robotic (Yor) population on it, and you need to get an organic Race population unit on that Core World in order to start getting population growth there.
If the world has organic Race population on it then research techs and build buildings or use Policies or Ideologies that will increase the growth rate there, or get rid of things that are reducing growth rates like Pollution, and possibly Crime.
You can also use one of the many Missions that you can get while surveying Anomalies, where you build something on a World and get one or two units of a particular Race there, e.g. Torian Academy, Manti Breeding Pool, Krynn Altar, Baratak Grove, etc. to get more organic population units on that world. Note once, you build it you can not use that mission again. However, before it is built, you can put that Mission into the build ques for multiple Core Worlds, and be able to build it on all of them and get the population units on those worlds.