I'm still building libraries et al in all of my cities. There's still not a reason to focus cities on some improvements over others. The specialization offered by selecting a unique building upon leveling doesn't narrow the focus of the settlement enough. And the fact that the same set of buildings is available for every city achieving each level further dilutes specialization.
As an example, if I pick Gallows for my lvl 1 improvement, I shouldn't be able to build an Almshouse later; instead, there should be another option still focused on reducing unrest primarily, with maybe a secondary benefit as well (I don't know, maybe Alcatraz Tower or something).
One way to examine this may be to remove all normal improvements completely (just as a test case), and permit settlements to gain an improvement only when they level.
Or, have each settlement start with rather generic options: workshop, marketplace, library, training ground, etc. Once one generic building is built, it has a branching tree of additional buildings that can be built. Also, each additional generic building costs a meaningful amount more than the previous.
Just my observations, and thoughts on fixes.
