Flow control?

One of the things we’ve been struggling with, even since the beta, when it comes to match making is the seemingly exponential server requirements for match making.  That’s why when we’d release a beta things would be peachy and then they’d go to the public, it would fall apart.

Well, yesterday they finally found the issue deep deep in one of the licensed network libraries was something called flow control. This limits the number of packets that will get processed based on a number of criteria.  There’s good reason for flow control but because this code is a “black box” to us, until we could meet with the developer in person and go through his code, we had no idea that you’d end up with scenarios where a player might only be processing 3 or 4 packets per second.

Believe it or not, 3 or 4 packets per second isn’t necessarily a bad thing in a client/server game. You connect to the server and they send back the connection info. Maybe that connection info is 10k. So it takes a second or two to get that info. It’s not that big of a deal if it’s 1 vs. 1.  It’s not even that big of a deal if it’s 2 vs. 2.

But naturally, in Demigod, people want to to 3 vs. 3 or 4 vs. 4.  And if you do that, well, very bad things start to happen because it starts to back up. You can tell it’s messed up because your connection window will not seem to make sense and it doesn’t – because it’s incredibly outdated.

If you have to get info every few seconds from say 8 people in a lobby, you’re going to quickly get backed up.  Now, until we fix this, you can resolve a lot of this by keeping your games at 2 vs. 2 or maybe 3 vs. 3 if the people in the lobby have decent machines.  It’s fixed on our end but we have to merge the fixed library into our code which, if you’re a developer, you know how carefully that needs to be done, even with version control.

But this problem I just described has been there since the beginning (i.e. February).  The only reason connectivity has gotten better is because we’ve thrown a ridiculous number of servers (I think there’s like 8 servers now dedicated to just handling requests) plus the work we’ve been doing for the past 2 weeks has been improving things.

BTW, the only reason why the day 0 update worked better for some people than the most recent release is because the day 0 update used NAT and that’s a lot slower (so a lot fewer messages). The faster the system, the more it aggravates this.

I’m going to ask our technology architect to do a write up on this once this is fixed.  I am heading back to doing more testing scenarios.

I plan to also write up an article on how games are made so people can get a better idea of how this sort of thing can happen in a major commercial game in the first place.

37,851 views 42 replies +1 Loading…
Reply #1 Top

Most developers / publishers would never come forward with this type of information to the public.  As always, appreciate the update.

Reply #2 Top

As always love the update BUT.... now that this problem was found when will it be fixed? Day? Week? 2 weeks? I am curious because i would really love to start playing the game instead of having to only chose a 2v2 .. or try pantheon w/ drops and AI :(.

Reply #3 Top

I want to go home and do more testing scenarios!!

Reply #4 Top

Also Frogboy will this kill Thrusday's update? As in will the proxys not even matter?

Reply #5 Top

Sounds like a possible cause of the "spiking" effect. Shame you discovered this after Yarlen pissed me off, already processing the refund! :D

Maybe when things improve i'll re-purchase. 

Reply #6 Top

I prefer a Coriolis meter and a Fisher E body valve for flow control, but that isn't always cost effective.  Sometimes I just settle for an orifice plate + DP cell and a cheaper Masoneilan 21000 or a segmented ball valve.

Have you considered doing this?

Reply #7 Top

Thanks for the info update. As you're no doubt aware, the transparency is really what's keeping this situation from getting really out of hand.

Hope things keep looking up.

 

Reply #8 Top

Try Kegel exercises.

Reply #9 Top

Cool info! i like it! Really appreciated Frogboy! 

Reply #10 Top

Quoting Mauxe, reply 8
Try Kegel exercises.
End of Mauxe's quote

L O L

Reply #11 Top

Goes to show, always write tests checking conditions in the worst possible scenarios :P

And of course, that fixing one problem, usually results in revealing another! :D

Reply #12 Top

Im sorry to say ... i dont appreciate this , not that your coming out to tell us , but i really dont feel like playing pantheon aI /drops matches or 2v2 , 3v3 custom games .

 

theres always a new problem coming up , nothing ever gets fixed , so many times have i read your update that says , its going to be fixed in 2-3 days , and now i have more issues then when i bought the game.

 

Im sorry frogboy , but my doubt is spilling up from my gut , problems might have been solved but now i cant play pantheon anymore, without playing with an unfair team every match, and i cant even dream of matches bigger then 3v3 ? , i thought this game was made for 5v5 and the other sized was a nice addition :/ .

 

 6.6 has not been released yet , you still have time , just make that time count man

Reply #13 Top


I plan to also write up an article on how games are made so people can get a better idea of how this sort of thing can happen in a major commercial game in the first place.

End of quote

 

Yes!

Want to read!

Please write!

Reply #14 Top

I remember speaking briefly to Frogboy in chat yesterday. Let his be a lesson to you all! Get jobs and invest in better computers! ;)

Reply #15 Top

Quoting XSive_Death, reply 5
Sounds like a possible cause of the "spiking" effect. Shame you discovered this after Yarlen pissed me off, already processing the refund!

Maybe when things improve i'll re-purchase. 
End of XSive_Death's quote

 

Mmm... I hope this fixes my spiking in 3v3+ games, but I have my doubts, seems completely unrelated.

Reply #16 Top

"Licensed network library"? So you actually paid for the piece of code that cost you so much time/money? Oh, the irony...

Reply #17 Top

Quoting bug, reply 16
"Licensed network library"? So you actually paid for the piece of code that cost you so much time/money? Oh, the irony...
End of bug's quote

That's common business in the industry

Reply #18 Top

Didn't anyone think at some point that hey we could have had a V8.

Reply #19 Top

Quoting Gabberkooij, reply 17

Quoting bug, reply 16"Licensed network library"? So you actually paid for the piece of code that cost you so much time/money? Oh, the irony...

That's common business in the industry
End of Gabberkooij's quote

 

Tell me about it... Jus today we had an issue with some JBoss library not releasing sockets in a timely manner. But at least we don't pay for it and we made sure we check for these problems before releasing.

 

PS No, we're no better than Stardock, we're just working on another stupid webapp so we have it easier.

Reply #20 Top

Moral of the story: never license code without having source access!

Reply #21 Top

Quoting AngryZealot, reply 20
Moral of the story: never license code without having source access!
End of AngryZealot's quote

Quoted and underlined.

 

Reply #22 Top

Hmmm...So this was the root of all problems? Maybe it will fix my astronomical lag spikes in >3v3 matches. I hope...

Reply #23 Top

Moral of the story: never license code without having source access!
End of quote

 

Its still best to get the original dev involved even if you have the source code. If you are not careful you will end up with a game like Star Wars Galaxies. Putting the   CU/NGE aside for a minute the amount of different people that they have had over the years screwing around with the code have left said code a mess that causes the devs now all kinds of probems.

Often they have no idea what a previous dev has done and when they try to change it it causes a bug or another unintended change some place else.

Thats why its always best to get ahold of the guy that coded it if you can.

Reply #24 Top

The license code is raknet.  Their software suck.

 

Looks like no 50% coupon until this gets fixed.

Reply #25 Top

Hmmm... And I always thought flow control was on the client side. You learn something new every day...