When I log onto ICO, and view the info screen for me in the lobby, my latency is always pretty low (30s, 40s) but when I go and try to play a multiplayer game, I always lag a lot, and my latency fluctuates from being as low as 40 to in the 7000s (this changes over a couple of seconds). Obviously I can't play a game with that much lag, nor do I want to do that to the other players, so I quit. Don't worry, I haven't tried to played more than 4 times, and each time I did something beforehand that I thought would fix it.
I already did all of the stuff in the Monk's guide, and I tried closing any non-system processes that were taking up a lot of memory, but I still can't figure this out. I have two 2-core processors, 2gb of memory and an Radeon X1900 XT video card that should definitely be able to handle this game (and it does it very well in single player). I also have a very fast cable connection, and live in southern Connecticut, if that makes a difference.
Can someone please help me out? I love this game but I'm already starting to get sick of playing against the AI, and if I can't play multiplayer I'll probably have to abandon SoaSE. Any help would be impossibly appreciated. Thanks.

EDIT: Forgot to add that, as far as I know, I don't have these kinds of lag problems with the other online games I play (but they're all server-based, not p2p)