I think it does turn in to your territory.
I recall one game where I was lamot at the % level to get the Influence victory, and built a couple Influence bases and found myself over it. However, it could also have been thatthe population of my planets had grown during that period to male it.
A test would be to bild several Influence bases, check your % towards Influence victory, save game, then destroy the bases, and check again the next turn to see if your % had dropped.
If you do that, please post the results for confirmation!
