I'm not the best player of the game. There are those who do better than me, but here is my meager attempt at answering your questions you posed.
1) How much of a challenge AI can provide? On normal and highest setting? Does AI use variety in strategies, or plays the same every time?
The AI has several different personalities and most of their actions are based on good random choices. The map is random so the AI cannot play the same way each time. There are isolationist type AI (I've got my cave and it will be POWERFUL), and there are expansionist type AI (grab everything because it's MINE). These type of AI are randomly assigned to each of the oppenents and you cannot always tell which you will get when you play the game.
2) How are factions different?
The manual has an in depth description of everything you are looking for here. But they are different, very different.
-Are they only different in the traits/access to traits/items/spells they have?
It seems you covered everything here, but they also look different in style. Kingdom / Empire have different looks altogether. Then you have color distinction between the factions. Yes they do have different traits.
-Do factions have different characteristics in the basic stats of units?
Yes, each faction is defined from a race. The race of people have particular abilities. Each of the units built from this faction use the racial properties.
- Do factions have unique things to them apart from traits/access?
Some races have unique equipment only to that race. For example Kraxis make 1 handed spears, no other race can make these items.
- Do different traits/abilities have enough impact on the gameplay to feel different?
You cannot play the game as one race and not feel your playing the same race as another. Each play style is distinctly different in style. You can play each race the same, but that may not be playing that race to it's full potential.
3) In a sandbox experience, what is the incentive to replay the game using the same faction, for example?
The game is designed to be completely random. Thus, the traits for the same faction is the same, but the playstyle must be adaptive depending on your start. Although, you may have the same traits, the tatics you used in the previous game do not work well in the new game because say you have less access to metal, but more access to crystal. Then your troops will be more magical in nature, and thus your playstyle must be more adaptive to this end. This means that if you are playing a less magical race you have added a new challenge by trying to utilize your traits in a different manner to be successful.
More importantly, if you are asking about replayability in general, you do have the ability to customize your sovereign for a given race. Also, any units you design the AI will use and somewhat effectively. Also, the AI will use any customize sovereign you make because the AI was designed to be able to play the game. The AI designer is also going to continue to upgrade the AI to be better and better in future updates... once he learns more peoples play styles and adds counters to those particular gameplay mechanics.
4) From the standpoint spells vs unit strength, how is the balance implemented? Does endgame not leave strong spellcasting faction prone to mass upgraded armies?
A spellcasting faction is not uncapable from building upgraded armies to defend themselves. In fact, most weapons with elemental damage leave some of these armies vunerable if the other faction didn't provide the appropriate resistances from this damage. The problem with building a super troop that is resistent to every form of damage is costly, and also limited by a units strength. There are also strategic spells (which the AI is not the best at using presently) that can stop armies in their tracks for a few turns. Also, there are spells that will scatter forces on the strategic map, then allow you to attack each invidual piece, while the oppenent trys to regather them together.
The balance is pretty good as far as things go there and there are multiple ways to win the game. Leaving players with options on the gameplay.