Your right about inf SB's being used to flip planets to your side.I find If other's build inf SB's near your world(s) it has little effect if there deep inside your territory or SOI (sphere of influence),
how ever if your colonies sit on the border with a race which has stronger influence than you or is actively trying to increase its inf by building SB's I find that researching the various cultural and influence techs helps a little I also build SB's and upgrade them after I have done the research, this helps keep enemy inf near my border colonies down.
If you can early on in the game build the restaurant of eternity it gives a 20% inf boost overall. If you attack an enemy inf base it will be a declaration of war, and if that race has allies they may attack you as well and your relation's with other races may suffer somewhat. so don't rush with guns blazing fight inf with inf its long process but can be more effective than all out war.
The other option is get another race to declare war on them if they do then star-bases are usually the 1st to be attacked.
I'm not so good with a cultural strategy as i favour war and manipulation and the occasional bit of genuine diplomacy as my general strategy, but I hope what little I do know helps your game. Browse the forums they have a alot of hints and tips regarding inf and "culture war".
