Short answer: Yes, they can flip it.

Longer answer:

Once you colonize, the planet will sit for a turn or two before opposing influence kicks in. If you are leaving the world as a colony, you're likely to lose it. However, if you're setting it up as a core world, you have an opportunity to get that world pumping out influence faster than your opponent's influence can take the planet.

When you zoom in enough, you'll be able to see your regional color forming a tight buffer around your planet, and a mouseover will reveal the percentages of control each faction is exerting on the surrounding hexes. If you are winning the influence battle, you'll see your numbers going up on your hexes and those surrounding them, and your influence will slowly expand.

You can use influence boosters (say, +500 to influence) to increase the immediate buffer, and more enduring boosters (of the +2 to influence sort) to help expand your influence in the area.

Colonies, as opposed to core planets,  can also receive these boosts as well as civ choices (such as +1 to colony influence) to punch holes in weakly held enemy territory.