There's quite a noticeable step change at season 100, where modifiers go from probably positive ("we are still settling") to probably negative ("our ambitions are rising"). In my game the modifiers went from +1 with Magnar to -2, and with Capitar from -2 to -6.

One of the things which changed in 1.5 is that they reduced the positive modifier for being from the same side, i.e. Kingdom or Empire. Prior to 1.5 it was fairly easy to be on good terms with factions on the same side as you, especially before season 100.

Reading the change log for 1.5, the intention was that there would be various modifiers which would help you drift back towards Neutral. I'm not convinced this is working as intended, as it seems to be fairly easy to get -2 modifiers, which will apparently make your relations one level worse every ten turns or so. One slight quirk is that because relations don't seem to change for ten seasons, regardless of modifiers, "first impressions last". In my game relations with Capitar stayed at Warm for ten seasons despite modifiers going negative two turns after meeting them. I assume the same would happen if you started off Cool; even if you made modifiers positive they wouldn't change relations with you for ten turns.

It depends to some extent how you're playing, but if you haven't made contact with AIs before season 100, then you may well find them rather aggressive.