PDA

View Full Version : The good and bad of combo packets



codepig666
02-05-2003, 08:38 PM
---edit---
After I posted this, I noticed that fee had already posted the same thing with even more detail.

While much of this is redundant, some of it is still useful I think. If not, just flame it a few times and I'll kill the thread.
----end edit---

I thought I would throw out a few thoughts on the concepts of the combo packets for people to think about and or flame me for:

The Good:
The network is a bottleneck. As the server makes its rounds to send data to any given client, it frequently has more than one packet in the queue.
In the old model, it would pull packets off the queue for a client up to the point where it reached either the end of the queue or some predetermined threshhold, sending out each packet one right after the other.
In the new model, when possible, it takes the queue'd packets and processes them into a combo-packet so that it only has to perform one network send per pass.
From a SERVER perspective, there is zero change in processing efficiency: The same number of packets will be processed off the queue as in the old model. There is a gain in network efficiency from the fact that what was previously a group of packets is now a single, larger packet.

The Bad:
The network is a bottleneck. As the client receives packets, it opens and processes them one at a time.
The natural latency of networks provided a natural smoothing effect. For example, if the server sent out 10 position update packets in the same 100 millisecond period, it is quite likely that the client received and processed them over the course of half a second or even more.
In the new model, all 10 of these small updates arrive in a single packet. They are processed in far more rapid succession than if they had arrived the old way.
Think of it like this: If you watch a movie at 30 frames per second, it is smooth because you view each frame for 1/30th of a second.....but if you watch the same movie with all 30 frames being displayed in the second half of each second, it would be bizarrely choppy.

Because the smallest packets are the most likely candidates for combo-packets, things like positional updates (a small packet) will be the hardest hit.

I really don't know what they could do to compensate for this effect. The natural answer would be to include timing information and then have the client space out the changes, but since players are already contending with lag, this additional delay would add to that pain. It would give back the smoothness but at the cost of leaving the player further behind the action, time-wise.

Perhaps the smarter approach is to measure the queue depth and only begin combining packets if it passed a certain threshold which would be indicative of a raid environment. That way people soloing or in regular groups would never experience the new problems while those involved in raids would reap the benefits.

Peace

compuboy86
02-05-2003, 10:53 PM
Nicely written.

Forward: SOE

Poncho
02-05-2003, 10:58 PM
If I was a Dev and had your knowledgebase, my next post would be something like this:

SEQ Dev ---> Hands resume to SOE --> You work with us, we'll work with you --->

WE ALL WIN

cattj
02-06-2003, 09:30 AM
This is a good example using the movie, but some of these packets are not visual, in that the packet and its information do not directly affect what you see. Think of it as the interlaced frames in the movie example. If you where to do a pull-down to remove the interlacing you wouldn't notice a difference because the framerate is identical. However if you configured your pull-down improperly and took out key frames you would notice jumping and stuttering.

Its all a matter of priority. Certain packets NEED to get to the client and be processed inside of a certain timeframe. You are simulating an activity that on the server is happening in realtime and in order for it to look believeable it needs to be timed properly on the client. For the data payloads that don't need this time the multipackets work great. For data payloads that require this timing to look "believable" it is important that they get more precedence in terms of being sent and processed in realtime (or at least in as much realtime as a network allows).

speedphreak
02-06-2003, 10:06 AM
There is a HUGE efficiency from batching packets together.

Network sends are VERY costly, even for UDP packets - and even (tho less so) with high end network cards that have processors onboard to handle some of the workload. Server side you must take into account every single person connecting to a world server, and how many UDP packets they are being sent. Less packets = much less server side processing.

That doesn't even take into account UDP packet headers that must get added onto every packet (and they get drastically reduced when concatonating packets).

The tradeoff is always efficiency/responsiveness. By batching packets together you need to decide on cutoffs - how long to wait for more data until you send (like Nagle), or how soon you decide you have enough data already and must send it, coming back to process the rest after.

throx
02-06-2003, 03:56 PM
By my measurements, the network traffic is about 30-40% of the size it used to be. I haven't noticed any glitchy movement of NPCs to talk about either so I'm guessing the movement code is working just fine. The movie analogy is actually incorrect because EQ sends velocity and even turning data to the client as well as positional data so as long as the mob doesn't change direction then you won't notice anything client side. From a quick look at the packet information, it's on a pretty quick accumulation timeout anyway so any glitches will be minor.

The massive difference you haven't touched is that around half of SOE's cost for running Everquest is the network charges for the amount of data they transmit (source: CNN interview). By reducing that to 30-40% of it's original value, whoever wrote this code has just made SOE around a million dollars a month richer. No company is going to argue with that.