I usually forget about upgrading outposts because they are disconnected from your city building queues. Once I remember, however, I usually put in stables for the faster movement. If necessary, I will use High Tower in long empty expanses to reduce the number of outposts needed to get through an area. Occasionally I will build a Consulate if growth is really bad. The rest of the upgrades I leave out because they are so limited in their area that they are not worth it. They only work in extended battles along a static front or in chokepoints. As soon as the front moves, your investment is rendered useless.
I would like to be able to have a stable-like effect within a city's ZoC without having to build redundant outposts within the ZoC. Either automatically grant stable effects within city ZoC when you research the tech for stables, or have a city-buildable stable.
I also have thought about some sort of road building upgrade that will connect the outpost to the next nearest city or outpost, or upgrade the road to a better, faster version. This is particularly an issue when you capture enemy cities and your road building tech does not automatically connect them to your empire, or when it autobuilds roads through another kingdom instead of around. You have to play connect the dots with outposts to try to bring the endpoints of your roads together to get the roads where you want them. You usually end up with a gap between two road systems where there is no road at all. Something like a road level for each outpost, starting at zero, that you can upgrade. The road between two outposts would be the lowest quality of the two endpoints. Another option for "New Roads" that would tell the autobuilder to find a path to the next nearest faction outpost or city within a certain range where the travel time would be reduced by more than X% by a direct road. (please also devalue building roads on lairs and quest locations)
I would also suggest the option of linking an outpost to a different city instead of just the closest. This would allow you to direct resources to the place you want them (grain, materials). Perhaps with greater building cost the farther away the outpost is from the city.
Another suggestion for outposts is to have pioneers build them, but instead of being used up, have them teleport to your nearest city and be immobilized for 10 turns. (i.e. they take time to build the outpost and then return to the city for new supplies) The 30 pop cost for a pioneer doesn't make sense for the building of an outpost. Especially when Pariden can plop them down so cheaply with Arcane Monolith.