So what exactly is it, how is it calculated?
the sim speed is basically, how many more cycles of the game (one instant in the game world if you prefer) your computer can compute during a game cycle. So theorically anything above 1 is enough, but since there are far more units at the end of a battle, if you start with a sim speed of 1 or 2, you are likely to experience some lag in the final moments.
Why does changing the graphics settings appear to have no effect on it?
It is because your graphic card doesn't limit your hability to run the game, it can also come from your processor or your ram.
Also, what difference does it make to me and other players what my sim speed is?
If anyone has a very low sim speed (typically 0), everyone experience major lag, it is either that or desync everyone. It sucks but if someone has a low sim speed, he will ruin the game for everyone, and the devs can't do a thing about it (well, ultimately, they can try to develop algorithms that need less computation power, but demigod is already optimised enough)