So if it took twice as long how would that change things?
It really depends on what else is available for your city to do, and at the start of the game you really don't have that much that you can build. I will say that increasing the amount of time that it takes to build the Tower will increase the value of getting the Cleric, Bell Tower, or Logging Camp (if any are available) before the Tower, as any of those can reduce the time taken to build the Tower and provide more of a construction time reduction (in absolute terms, not relative) the longer the Tower takes to build, though I could not say off the top of my head whether or not doubling the build time of the Tower would make delaying its benefits for those of the three listed structures worthwhile (especially since that judgement is dependent on the player - what's worthwhile to me isn't necessarily worthwhile to someone else; I'll already build Logging Camps and perhaps Studies before the Tower if I have them available, anyways).
Getting that growth bonus from the Tower early means you reach level 2 faster, and you can build pioneers earlier and more frequently. The Tower also provides a comparable unrest reduction to the Cleric, because your first city gives you 3 unrest due to having 1 city in your empire, which is canceled by the Tower, while the Cleric reduces unrest by 5. I might build the Cleric first because it builds a bit faster, but on the other hand I'm not likely to get additional structures that I want to go up quickly early enough that the order of construction will matter much. Bell Towers are more tempting, if available, but on the other hand the unrest levels are probably fairly low in the very early game anyways, since if you're like me you won't be taxing your empire at more than the 'low' level (and more likely you'll have taxes at 'none') simply because your empire is so poor that it doesn't provide an appreciable amount of income anyways; the Merchant likewise goes out the window for tax-related reasons. Piers provide too small a bonus to consider unless you just really need a bridge across the river, and moreover require that you're actually next to a river before you can consider building them.
Logging Camps are both useful enough to be worth building first and can have issues if they aren't built first (because they have to be built on a forest tile, not the city tile that used to be a forest or just in an arbitrary position in a city whose hub was adjacent to a forest tile when founded) to go before the Tower, but you have to have settled next to a forest in order to get it in the first place. Shard Shrines are variable, depending on how much you need early mana or the shard demons (if you have binding), but it requires a specific faction trait, and the usefulness of very early mana boosts is debatable, particularly when you can most likely cast Meditation on your first city and get more mana; the elemental shard you get from the shrine is also of variable usefulness, depending on what type of shard it is and what types of magic your sovereign and early champions have. Studies are a reasonable first build, but like shrines are only available through a specific faction trait. Bazaars are not useful until you have a reasonable amount of money production.