DEVS: Locating offsets - part 1
Preface
It has been awhile since anyone has updated the "Locating Offsets" sticky, and since I cannot edit it directly, but do not wish to unsticky it, I will add a new "Locating Offsets" sticky post. You can still view the old one, which is now a sticky in this forum that was renamed to "DEVS: Locating offsets - part 2".
About MySEQ
MySEQ's server is a native Win32 console application written in C++ using the x86 Microsoft Windows Platform SDK. It runs on the same machine as Everquest, and operates by 'peeking' into the EQ process and looking at it's memory contents. It does NOT change anything in memory, and thus does not alter the way the game plays in and of itself.
Offsets 101
The purpose of the offsets is to provide an easy way to update the server each time a patch comes out, without having to make a new binary release each time. Sometimes the game changes so dramatically, a new server is required. Sometimes we just want to add new features to the server. However, most of the time you can use the server as-is, and just get new offsets. The 1.17.0 server worked fine for almost two years without the need for an update. That's pretty good.
The offsets live in a file called myseqserver.ini. Each entry lives in a section, which is enclosed with square brackets ([,]), an entry name, which is the value to the left of the equals sign (=), and a value, which is the value to the right of the equals sign. Each entry value can be decimal, hex or string values. I generally like to keep values in hex when I can, because most RAM editors default to displaying in hex, which minimizes the hex->dec and dec->hex conversions one needs to do. Please note that hex ini values only work from v1.18.0 and up. Before then they needed to be stored as decimal.
Critical Primary Offsets
The main goal is to find the 'static' pointers. These are values that do not change just because you shutdown and restarted EQ. These static values are critical for MySEQ to work. There are four critical static offsets that MySEQ Open needs to work, and they are located in the [Memory Offsets] section:
ZoneAddr - this offset is a direct pointer to the 'short name' of the zone. We use the string located at this offset to determine which map to load. I may refer to this as pZone.
CharInfo - this offset is a pointer to the player information. It is used when determining which spawn (a spawn is just a list of all entities in the zone) is the player. This comes in handy when you need to center on the player, or for determining 'con colors'. It is also used to draw the range circle around the player. I may refer to this as pSelf.
TargetAddr - this offset is a pointer to the currently targeted spawn in the game. It is useful for when you want to center on target, or just to see the 'target information' in the 'Target Info' box. I may refer to this as pTarget.
SpawnHeaderAddr - this offset is a pointer to the linked list of spawns in the entire zone. It is a doubly linked list (forward and backward). As of the writing of this note, the list starts with a valid next pointer, and a NULL prev pointer, and the end of the list seems to have a next pointer back to itself. I sometimes call this pSpawn, pSpawnlist or pMobs.
Non-critical Primary Offsets
There is one more offset for ground spawns(ItemsAddr), but I have not seen this working in quite some time. Some zones have items that drop on the ground, which appear as small bags. This used to be the pointer to view those items, however these structures appear to have changed, and the static pointers to find them have yet to be re-discovered. Having a non-zero value in here can cause lockups, so unless you KNOW the correct value, just set it to zero. The new servers will skip over checking for ground spawns. The old servers will check the memory at offset 0x0, which will fail and give the same result, although at the cost of a few cycles. The new server improves the speed.
ItemsAddr=0x0
RAM Editors
In order to analyze the EQ process space, you will need a RAM Editor. So far I have come across three.
1) WinHex - probably the best RAM Editor you will get your hands on. You can get it at http://www.x-ways.net/winhex/index-m.html
2) WinHack - a nice little RAM Editor, but it is hard to find these days. It does have a 'training' feature, which allows you to monitor a memory block for changes. I wish WinHex had such a feature.
3) Powerhex - a nice looking editor, and cheap. However it seems to choke on large applications. Searching takes a long time. www.bo-solutions.com
Accessing the EQ Process
In order for your RAM Editor to view EQ's process space, you need to run it at a higher privilege. Much like cron jobs in Unix, Windows has an 'at' deamon that is running all the time. To see what is currently scheduled, just open a CMD console and type 'at'. Jobs spawned from the AT daemon run at the highest privilege, much like cron jobs are spawned with root privileges. By running your RAM Editor via the AT daemon, you will be able to 'see' all the processes in the system, including EQ.
I wrote a small batch file, called ramedit.bat to make it a little easier for me, it looks like this:
Code:
@echo off
cls
:HELP
time /t
if not "%1" == "" goto MENU
echo Usage: ramedit (time to start) (hacktool number)
echo 1) PowerHex
echo 2) WinHex
echo 3) WinHack
goto DONE
:MENU
if "%2" == "1" goto POWERHEX
if "%2" == "2" goto WINHEX
if "%2" == "3" goto WINHACK
goto HELP
:POWERHEX
at %1 /interactive "C:\Program Files\PowerHEX\PowerHEX.exe"
goto DONE
:WINHACK
at %1 /interactive "C:\Developer Tools\Hex Editors\WinHack\WinHack2.exe"
goto DONE
:WINHEX
at %1 /interactive "C:\Developer Tools\Hex Editors\WinHex\WinHex.exe"
goto DONE
:DONE
Basically you run it with no arguments and it will tell you the current time.
> ramedit
1:25 PM
Usage: ramedit (time to start) (hacktool number)
1) PowerHex
2) WinHex
3) WinHack
Then you run it a second time, this time with two arguments, as indicated. Convert to military time, but add 1 minute.
> ramedit 13:26 2
1:25 PM
Added a new job with job ID = 1
Within a minute the editor will start up.
Note: You will need to edit the script to point to your own RAM Editors. Also, when running the second time, make sure the 'next minute' has not elapsed already, otherwise you will be waiting for 23 hrs and 59 minutes for the RAM Editor to open.
Hunting for the Zonename
Undoubtedly the easiest offset to find is the pZone offset. Just do a string search for the shortname of the zone (look in you maps folder for a list of zone shortnames). In general, the static pointers will live between 0x00800000 and 0x00A00000. Dynamically allocated memory will appear much higher, like over 0x1000000, however some may drop as low as 0x0C000000.
Once you find the zone shortname, just record the offset value into the ini file and reconnect the client. You should see the client load the correct map.
Hunting for the Target
The next thing we want to do is to find pTarget. More importantly we want to find any spawn object in the linked list. Once we find one, I have added a 'walker' debug mode which will walk up and down the linked list. When it reaches the end, it will scan the static area for the pointer and tell you where it is. To find the pTarget, select a unique target (like a banker) from inside the game. Then scan the memory space for that name. Replace spaces with underscores and add on '00' to the end of the name. So to find "Buck Johnson", scan for "Buck_Johnson00". When you find it, you will need to then 'backup' by NameOffset bytes. You can view the current value of NameOffset by looking at the NameOffset entry in the [SpawnInfo Offsets] section. Once you have backed up that many bytes, write down the hex address you are sitting on. You will need to scan the static area for that value, which brings me to my next topic, endian issues.
Sidebar: Endian Issues
Intel/AMD processors are all little endian. PowerPC processors are big endian. Do a google on 'endian' if you want to learn more, but basically because we are dealing with LE boxes, our pointers are going to be ass-flopped around.
When a LE processor grabs a byte from memory, everything is fine. If your memory space had a DE AD BE EF in it, and you read that in as a char array, you would get DEADBEEF into the char array. However, if you read that same value as an int, it will flop it, so the value you actually see is EFBEADDE. So when searching for pointers, you will be looking for the ass-flopped version of the value, and not the value itself. WinHex does the ass-flopping for you when you use the 'Integer Search' feature, and it even pops up a nice message telling you that it found your pointer, but it found the ass-flopped version... which is correct.
If you located the pTarget in the static area, then put that value into the ini file and restart the server. If you found it correctly, you will see the value at that static offset change, and if you follow the pointer into deep memory, you will see the new current target's information.
Hunting for the Ends of the Spawnlist
Once you have pTarget, you can use the new debug modes to assist in finding the other offsets. In the [Debug] section, set the Flags=0x1000 and restart the server. Make sure you have something targeted in the game, and then connect the client. You will see a ton of debug messages as the server walks the spawnlist. At the end, it will scan for the 'last good pointer' in the static area. If it finds it, try using that value for the SpawnHeaderAddr. It also, as of recently, seems to be the same value for pSelf, so set the CharInfo value to that as well.
Secondary Offsets
Secondary offsets are all the other offsets shown. Once we know the pointer to a spawn is valid, we will want to get other good information, like the spawn's X,Y,Z coordinates. You can use the other debug modes to assist you with that. Secondary offsets do not change very often, which is good. Read the comments in the ini file for more details on the other debug modes.
pSelf variations
One thing I have noticed is a variation in the way one finds pSelf. For a long time, the CharInfo offset was a pointer to the top of a Character Info structure, which had a lot of data about the character. One of the things it had was a pointer into the spawnlist in which the spawn pointed to was the player. That is what the [CharInfo Offsets] section was used for. We would take the static offset [Memory Offsets]:CharInfo, look at where that was pointing to. Then walk down [CharInfo Offsets]:SpawnInfo bytes to find yet another pointer. You then follow *that* pointer to actually get the spawn information, like X,Y,Z coordinates, race, class, etc. Recently I have found this 'indirect' method was not needed. There are direct pointers to pSelf. To use 'direct' mode, you need to enable bit 1 in [File Info]:SchemaVersion.
[File Info]:SchemaVersion
Originally SchemaVersion was used to just swap the X and Y values of the offsets. I am not sure why they didn't just swap them in the ini and leave the server out of it, but the point is nearly moot. I decided to use it as a 'flags' field for when we need to operate the server in different modes. The current flags are as follows:
Bit 0 - Enable X/Y swap. The XOffset becomes YOffset and vice versa.
Bit 1 - Enable pSelf direct mode. In this mode [CharInfo Offsets]:SpawnInfo is ignored (see above)
Bit 2 - Enable prev/next swap. This is probably an oversight. I think just swapping the ini values would have fixed the problem without a server change.
I will probably be removing Bit0 and Bit2, since they appear to not be needed. However Bit 1 might still rear it's ugly head in the future, so I will leave that one available. For now, leave it enabled in the ini.
