New Test Release

[cavemanbob]

This is a test version with some simple new client pull code, in contrast to the old DOS server spam model It may or may not work, but that's why it's a test. This will hopefully fix some of the random stop working bugs...


http://alteria.sf.net/myseq-test.zip
http://alteria.sf.net/myseqserverc-test.zip
http://alteria.sf.net/seq-maps.zip

Server Changes:
* Update command line parameter now does nothing. This is handled on the client.

Client Changes:
* New Update option in the Options Dialog. This is the delay to wait between receiving a stop packet from the server and sending a request for the start. If this is set to 0 it will go as fast as it can, limited either by the client processor, the server processor or the network, whichever is slowest. It can update really fast at 0 though.

And of course if there are problems, please report them

[MrDoh]

Caveman,
I'm sorry to double post this, but now that it's in an old thread (guess we were posting at same time) I didn't know if you would see it, and kinda want to know if my logic is anywhere close.

Just a thought about this...

----
Lemme know if it won't work, but there should be a "formula" that can be worked out that will.

You set how fast the server reads the memory updates. I see no usefull purpose on polling the server any more often than the server itself updates. Would it possible, (or even beneficial at all) for in one of the very initial packet exchanges, have the server send its update speed to the client, and let the client start with that poll the speed (this is where that formula comes in... maybe have to add a few ms for the server to get everyting updated before it sends it). Then, if the users wishes, they can modify the update time more to their liking. If 250ms is to fast to send the packets out (1/4 second shouldn't be too fast I wouldn't think) then what's the point of having the server read that fast. And if the client is set to update faster than the server, you are causing unnecisarry network traffic because nothing will have changed because there hasn't been an update. (read duplicate packets)

Just some thoughts,
Mr. Doh

[cavemanbob]

Interesting idea, but in the current scheme the server won't poll more often than the client does. In fact the memory reads are directly linked to the client's requests until I work out a better solution. I'll think on this though, it could be a way to avoid at least some client -> server communications.

[cavemanbob]

Out of curiosity has anyone that was having trouble with the serevr locking on sends had this version work better?

[MrDoh]

Well, here is what I have found. So far, it has fixed my intermittant lockup problems in pre-LoY zone. (At least it hasn't locked up yet, but I have not been in game very long.) I do think that problem is fixed, based on what was thought to be happening.

Now, back to the dreaded LoY lands hehe.

Not much has changed. I zoned from steamfront, and it didn't even finish clearing the spawn list! It also didn't change the map.
If I start it up from inside gunthak, it does still load the map, still shows the location I am standing when I started the client, and at present there are even 3 skittles other than myself on the map. There is nothing at all in the spawn list.

When I "stop" and then "go" the client, nothing changes on the client side. (The server side still does what it should.) It doesnt update my position, the same 3 other skittles are all still there and in same spot. Well no sooner than I say that, this time when I pressed stop and then go, it did redraw part of the map (over part already still there), it did move me to mey new loc, and there are a few more skittles. Still nothing in the list, and not updating loc, etc.

I am going to check and see if you left the debug features in the current build. I'd like to capture the spawn list of gunthak from the server and see what is there. Is it possible that *something* in the spawn list of gunthak is messing with something in the client? I wouldn't think so, but there is no telling what SoE may be trying in LoY.

Anything else you can think of that I can do? Is there a way I can dump 1 set of packets to a txt file for reference, then maybe dump a non LoY zone and do some down and dirty comparisons?


Mr. Doh

*edited for more detail and more testing

[cavemanbob]

I think I'm just going to download LoY over the next day or 2, it'll only take 9 hours so I'll just leave it at night. There's a bit of code in the server where it sends the spawns to the client to dump the first mob to a fire called mob0 commented out. It's fairly easy to adapt to dump all of them, just stick it before the send and change increment the number in the filename. Unfortuanately the debug code was removed, but it was just some printfs around the read/send blocks anyway.

[MrDoh]

CavemanBob, please check your PMs if you would.

thanks

[MrDoh]

Most of my client lockups continue... My lockups (hard freeze, end task) seem to mostly (but not absolutely always) occur when I'm actively working with the interface, not when I'm running and not actively using it and seeing everything update.

I can still say that the pull solution you have implemented seems to work very nicely.

It's still interesting to me that the client doesn't lockup with the LoY problem. You can still "stop" and "go" and use the menus, etc. It just doesn't give a spawn list, or continue to update the client after that initial spawn list never happens. But yet I have seen the spawn list with a hacked up debug version.

For what it's worth,
Mr. Doh

[cavemanbob]

I've got to admit, I have no idea what could be causing lockups by using the UI. Is there any particualr one that gives the most trouble?

[Alfred]

My feedback on new client server handshaking:

Again, I'm using a 300mhz PII for the client.

1) If I leave the client alone it will work fine for a long period of time before locking up.

2) If I start changing things, clicking or doing many other menu related tasks on the interface, it locks up.
A) I attribute this to buffer overrun. The client sent out an "its ok to send" packet when in fact I started clicking and took cpu cycles away.... I have no idea if this is in fact the truth.

3) The very first versions that did not have the speed lines updated the screen much more quickly. In fact, before you made this test version with the handshaking, I couldn't get it to work no matter what delay I used. Now I realize that all the extra features are simply to much for my little old box to handle hehe. It looks like I've got a delay of about 2500ms when in fact I've tried 0 through 1000ms .

soo... the hand shaking route is the way to go.. .Now I've either :

1) Got to upgrade my hardware (on my 4th machine /sigh)
2) Go through the code and see if there is a way to introduce a toggle for extreme bare minimium functionality for stupidly slow PCs. I'll have to think on that one....

Anyhow, great work so far. I don't know if we are out of the woods yet with the lockups but this is a step in the right direction.

cheers

[MrDoh]

Well, I think Alfred has some very good points and observations.
The lockups do mostly seemto happen for me when actually clicking buttons, map skittles. The zoom and X, Y buttons do it to me a lot, but then again, I use them a lot. Sometimes it happens moments after you start the program, sometimes it may be hours, depending on what you are doing with it. But just now as I type, I tabbed to a different window (it wasn't active to begin with) and it froze. It was never even the active window, much less clicked on.

As for my hardware and O/S, the client is on 440mhz PIII with 128 mb ram running NT4SP6a with .NET framework 1.1.

Runs very smoth on this machine, even at the default 250ms (fast as I think anyone really needs hehe). Which as said in other posts, it looks amazing, and real time!

What type of network packets are you sending? What protocol are they based on? How does the client-side pull actually work? What kind of packet flow control is there? What size packets are being passed, and are they being passed one at a time, and then the next update time, requested and passed again, or is it broken up by the different offsets, or are all packets a certain size?? What is a *maximum* packet size??
(I'm a network engineer on the hardware side by trade, with other interests in PCs, so everything is seen through slightly tinted glasses.)

Please let me know please about the packets so I can get them off the wire and look at them. My normal sniffer doesn't seem to understand them, and isn't letting me snag them... hmmm...

This is a wonderfull program and getting better by the day, we just still have a few things to work out.

[Dark]

I used the test client and it started fine but everything stopped moving after about a min. If i closed it and started again same thing happened.

My eq machine is a 2.66 adn client machine 1.7gig both with 512 ram.

I rolled back one version.

One thing that would be helpful is the version number in the top blue bar.

[cavemanbob]

Ahh, this may very well be a performance issue then. I'm DLing a .net profiler now so I can see what is sucking lots of cycles.

The communications happens as follows:
The protocol is TCP, UDP has less overhead, but is more of a pain to use. All sockets are blocking and because it's TCP delivery is guaranteed.

All packets are 96 bytes long regardless of function and are of the SPAWNINFO_SEND structure, this is just for simplicity. I'm not sure of the timeouts now that I think about it, I'll have to look at those.

Here's how the data stream goes:

Client: Packet request -> Server
Server: Start Packet -> Client
Client: Packet request -> Server
Server: Target Packet -> Client
Client: Packet request -> Server
Server: Player Packet -> Client
Client: Packet request -> Server
Server: Zone Packet -> Client
Client: Packet request -> Server
Server: Start Packet -> Client
Client: Packet request -> Server
Server: Spawn 1 Packet -> Client
Client: Packet request -> Server
Server: Spawn 2 Packet -> Client
.
.
.
Client: Packet request -> Server
Server: Last Spawn Packet -> Client
Client: Packet request -> Server
Server: End Packet Packet -> Client
Client: Sleep for n ms for a delay, then repeat

If there's packets bigger or smaller than 96 bytes something in the stack has broken them up or combined them, it's not really important. As another thing I just thought of, in some cases the sent packets might be getting buffered for a while on the server side... Combining all the little bits into a big packet is starting too look like a better and better idea.

I've put the version number in for the next one, I meant to get it in the last one, but forgot.

[MrDoh]

I don't know, you might could call it a performance issue, but I think it may be a timing issue. I've been watching, and my CPU only maxes out to 100% for about 4 seconds when you first hit the go button. Even when it gets the update, it only peaks at 30%. When using, say the zoom buttone, it uses no more than 45% or so, and only for the second or so it takes to redraw the map. It just froze on me, and it was at 48% cpu usage. When you really come down to it, I wouldn't expect the client to require a large cpu at all. We aren't drawing complex textures, or doing immersive 3D etc. No severely complex calculations, right? I would expect something like this to run like a top on a 450mhz machine. My client machine is only a P3 550 running xp /w 384mb ram, and it runs just fine with myseq server, outlook, several browser windows, etc, and the only time it if slows down EQ running with high graphics is in the bazaar and pok.

Oh, and the network structure looks great. Using tcp is definately the way to go here imho. One question, when the client recieves the packet, does it immediately dissasemble and parse it, etc, or how? Is it still trying to take care of packets that's already arrived while striving to complete processing on what it's got. Yes, that is set by the prefference as to how long to update, but if it isn't finished withe the last set, or it falls "X" number of packet sets behind, should we through in a warning, or a pause to complete processing? To paraphrase a line in "Four Rooms" , I'm not a bunny, and you're not a rabbit, so lets not jump ahead. hehe

I really don't know what the issue is at this point... but at the same time, I have no doubt we'll track it down.

Mr. Doh

[cavemanbob]

The client immediately pulls the packet apart before asking for another one, so the server shouldn't send another one till the client's done with the last one, so it shouldn't get ahead, but I'm not absolutely sure this isn't happening. I'm going to throw together a different version of the client & server to take one big packet anyway jsut to see what happens anyway though.