There are two common cases for loading:  Starting the game fresh and loading a save file, and being inside a game already and reloading a recent save from the same game because something went wrong.  Both times, loading is a rather expensive operation, because there are a lot of things that need to be set up in memory.

Saving, on the other hand, goes a lot faster than loading.  Which suggests a potential optimization:  When loading a game that you determine to be the same game as the current game in progress, (which can be determined any number of ways; the simplest and most foolproof would be to simply put a GUID in the save file header that gets generated at New Game initialization,) save the current game to a memory stream and then run some sort of diff to determine what has to be done.

This can save a huge amount of work.  For example, unless someone's been casting land-altering spells, the entirety of the terrain can be preserved intact.  If it's a recent save, such as an Autosave, being loaded, most of the cities won't have changed too much.  And so on.