Current Development Status and Future Plans. Your Feedback Requested

[cn187]

At BlueAdept's suggestion, I'd like to detail where I think we are with regard to ShowEQ development, my near term plans, and my vision for future development. I also want to solicit feedback from everyone to get their opinions of how things are, and where you think they should go from here.

Current Status:

The release of the first ShowEQ 6 release candidate marks what I would consider "code complete" for the Qt3 to Qt4 migration. Any changes in the near future will be limited to bug fixes to "shore things up" before the official release. Bugs reported in ShowEQ 6 that are not present in ShowEQ 5 will mostly likely be fixed before the final release (though there could be exceptions). Bugs that are in both ShowEQ 6 and ShowEQ 5 may be fixed now or may be put off until after the final release, depending on the bug. As bugs are found and fixed, we'll produce additional release candidates for general use/testing. Your testing and feedback here is crucial.

Plan for ShowEQ 6 official release

Once we feel things are ready and outstanding bugs have been addressed, we'll promote ShowEQ 6 to be the primary release. I'm currently expecting this to be around the first of the year (give or take), but this could change depending on the quantity/severity of bugs that are reported, how the CoV release impacts ShowEQ, etc. Once 6.x is promoted to be the primary release, 5.x will be deprecated. Afterwards, we'll continue to update 5.x with new opcodes/structs for a short period of time (maybe a few months), but there are no other changes/fixes planned for 5.x. As such, please plan to upgrade to 6.x before support for 5.x is completely stopped. (There will be one or more posts about it before it actually happens)


The road to ShowEQ 7 (and beyond)

Once ShowEQ 6 is released, I'll start working toward ShowEQ 7. This will move us from Qt4 (which is end-of-life) to Qt5. Qt5 will allow us greater flexibility with things like themes and interface customization (hello dark theme!), and should also add support for running on distros that use Wayland instead of X11. EDIT: Initial Qt5 support was added in 6.2, and continues to improve. ShowEQ 7 will target Qt6.

In addition to 7.x moving to Qt5 Qt6, I have a number of features and fixes I'd like to implement. These may not all make it into 7.0 (in fact, I expect that many won't), but then there's always the possibility of 7.x point releases or even 8.x...

(in no particular order)


1. Autodetect the network interface if it's not specified (or if the specified interface doesn't exist) Done in 6.1.0
2. Overhaul interface styles/themes, including more customization options Done in 6.3.0
3. Where possible, fix functionality that used to work but doesn't any more - Work is ongoing, with no specific release target
4. The ability to show/hide certain things from the map display (at minimum, corpses. But it might useful to be able to specify an arbitrary filter similar to the search/alert filters) Edit: Apparently, this is already possible and I missed it... The "filtered" type in filters/zonefilters will hides things rather than highlight them. Still, it might be nice to have menu toggles for pc/npc corpses - TBD
5. Multi-layer map support (to remove the need to squash multi-layer maps into a single layer). Done in 6.3.0
6. Implement per-zone preferences (for things that can change from zone to zone, like depth filter head/floor settings) - TBD
7. Add handling for some additional opcodes I've figured out, and maybe figure out a few more - Work is ongoing, with no specific release target
8. Internal code cleanup to leverage things that exist in Qt4/Qt5 but didn't exist when ShowEQ was written - Work is ongoing, with no specific release target
9. Improve the UI for adding/editing filters - Planned for 6.4

There are also some old TODOs (which I think were originally from Zaphod) that I think would be worth implementing if I can pull them off (#10 in particular is not going to be easy).
10. Specify structs like we do opcodes (via XML) to make it possible to update without recompiling - Planned for 7.0
11. Assuming 10 can be implemented, add the (opt-in) capability to automatically download the opcode/struct updates from SEQ site - Planned for a post-7.0 release, TBD
12. Split the UI part from the capture part, so everything doesn't have to be run as root, and to possibly allow capture on one machine with the UI running natively on another - Planned for 7.0

13 (added due to my previous edit) Better help/docs on what ShowEQ is capable of and how to do various things. - Planned for 7.0

Your feedback is wanted!

What do you think about the above? Love it? Hate it? Which parts? Have additional ideas? Have examples of things that used to work but need fixing?

Let me know! I'll use your feedback to adjust my TODO list and to help prioritize what I work on.

[fransick]

Thank you for creating a plan to keep SEQ going and keeping the QT foundation from obsolescence!

I am happy to help with #7... I have a long list of opcodes I've discovered along the way that had no use in the current implementation. Many were added to the zoneopcodes.xml file as a placeholder with the thought someone more ambitious might come along and want to make use of them. It's easy enough to update with current opcodes if/when that data is needed for new functionality or development.

As for #10, of late it seems the structs have remained fairly stable and they do not mess with them nearly as much as they did in Zaphod's days. That said, some variable length structs were moved out of everquest.h to read data from the variable-length data i.e. the charPlayerProfile struct in zonemgr.cpp. I've become okay with reading structs and adjusting them if that skill is needed.

Happy to help in whatever limited way I can!

[cn187]

I am happy to help with #7... I have a long list of opcodes I've discovered along the way that had no use in the current implementation. Many were added to the zoneopcodes.xml file as a placeholder with the thought someone more ambitious might come along and want to make use of them. It's easy enough to update with current opcodes if/when that data is needed for new functionality or development.


Awesome! We should definitely compare notes at some point.



As for #10, of late it seems the structs have remained fairly stable and they do not mess with them nearly as much as they did in Zaphod's days. That said, some variable length structs were moved out of everquest.h to read data from the variable-length data i.e. the charPlayerProfile struct in zonemgr.cpp. I've become okay with reading structs and adjusting them if that skill is needed.


Agreed. Normally it's really just the 3 structs that change, plus occasionally the spawnshell code when the size of the posData changes. We could start with those, and then worry about converting the others if/when they change.

[BlueAdept]

I think this is a good road map of the future of SEQ. Thanks again everyone for helping and keeping it alive.

[Spanners]

Awesome news (and hi once again, I awaken from my slumber :))
I'll have a poke about on the pi to see about getting this release built. Hopefully I can find one of them and remember what I'm doing :)

[cn187]

I'll have a poke about on the pi to see about getting this release built. Hopefully I can find one of them and remember what I'm doing :) FYI, there's a recent commit (that didn't make it into RC1) that should fix an issue with detecting Qt on non-x86 platforms. So you'll want to either pull the latest from the pre_6_0_beta branch of svn, or use the --with-qt-* configure flags in order to specify your qt lib/include/bin locations.

[Spanners]

FYI, there's a recent commit (that didn't make it into RC1) that should fix an issue with detecting Qt on non-x86 platforms. So you'll want to either pull the latest from the pre_6_0_beta branch of svn, or use the --with-qt-* configure flags in order to specify your qt lib/include/bin locations.


Noted, thank you.
I have located 2 Pi's so I'll see what the latest distro's offer in the next couple of days or so and hopefully will get a good run at it all on the weekend.

[Spanners]

Apologies: For clarity, Raspberry Pi, clean image of Raspbian/Debian on a Model b Rev 2 512MB


Initial notes.
Pulled showeq-6.0.0.0-rc2
Compile was unhhappy even when using --with-qt-libraries=/usr/lib/arm-linux-gnueabihf as it then went on to be unable to find Qt Documentation, and providing any --with-qt-docs options failed so I skipped the qt validation.

On running, x-forwarded to xming on my PC (as I did withe the previos version), it loads up and runs, client detection is good and skittles are alive.
The UI is very "laggy" for want of a better word.

sidenote: downloaded maps from eqresource, oddly they had very little content for brotherisland and a mapconvert still showed nothing interesting, so I copied my other map accross from the old pi and that works like a charm.

Map - fixed with import from other pi, dubious source or something ... tbi.

I'll give it a run to see what else occurs, but if you need any specific feedback on anything please ask and I'll see if I can arrange some :)

Cheers!

Addendum: After about 40 minutes the xming session was unworkable, but nothing really showed up in taskmanager suggesting it was having issues.

[cn187]

For X-Forwarding issues, see http://www.showeq.net/forums/showthread.php?7446-ShowEQ-6-Beta-Test&p=50652&viewfull=1#post50652 and http://www.showeq.net/forums/showthread.php?7446-ShowEQ-6-Beta-Test&p=50657&viewfull=1#post50657

[kkmonte]

CN187, I know we discussed this via email but i'll add one that would be nice if able. I'd love to be able to get the exp window working again, not sure if the information is limited to what is sent to the client, maybe it can do some kind of calculation if all the info isn't there, not sure if the zone zem is sent and could be displayed, so at level 81 for instance, I could test out some zones to see where the best exp is. Also is AA exp different zem or the same? I know AA exp is supposed to be something like the same exp to go from 50 to 51 or 51 to 52. If we need some help doing research on official EQ posts to put data together let me know. Now with the EXP bar going out to 3 decimal places, it should be easy enough to do some tests to see if we can figure out anything we need.

[cn187]

Sorry it's taken me a bit to respond to this... the past few weeks have been a little hectic.

Originally, they used to send raw XP numbers and the client would turn that into a percentage. As far as I can tell, they're now sending just the percentage that you see in the window, and all calculations for display are done server-side.

I don't know if ZEM is still sent, but if it is, it's not where it used to be in the stream because the values displayed by SEQ are obviously incorrect. I sort of doubt they're sending it, though.

Regardless, it should be possible to log exp values across zones, and compute effective ZEM once enough data is collected. For example, if you kill an even-con mob in one zone and get x%, but then kill an even-con mob in a different zone and get y%, those zones obviously have different modifiers. If we can do that in enough zones, then we should get a feel for what the baseline is, and an idea of what the modifiers are (in relation to the baseline) for various zones.

If we're really lucky, we might be able to use that data to find that they are indeed still sending ZEM in a packet somewhere, in which case we can make use of it. Otherwise, we can just alter SEQ to collect the necessary data and do the calculations, and/or compile the data and hardcode the ZEM values for the various zones. Except for hotzones, I don't think they change all that much.

If you (or anyone else) want to do some experimentation, that would be super helpful. Zone name, your level, mob level, xp per kill, and whether it's XP or AAXP would be what we'd be looking for, I think.

[BlueAdept]

From another site I am on, they tried to get a list of ZEMs but it isn't a fixed value any more. They have pretty much given up on figuring it out now. They originally started to figure it out by how much per kill you get for even cons, but with Hot Zones and patches that all changed from time to time. So one patch a ZEM of 1.5 might be 1.0 next patch. Lower zones may stay the same but it is too much work to keep checking them (especially since they are not popular any more) to keep a list going. I think now they can also change the ZEM by how popular the zone is.

[cn187]

Interesting... But not surprising, I guess.

[cn187]

If anyone wants to preview/test the stuff that I'm working on that will eventually make it into the next feature release, I've created a copy of my private dev branch at https://svn.code.sf.net/p/seq/svn/showeq/branches/cn187_devel

As I finish things, I'll test them a locally for a while and then push them to that branch for wider review/testing. They'll eventually get merged to trunk as part of a feature release, but I want to make sure they work for people other than just me. So if you want to track that branch instead of trunk, please feel free to do so. I'll try not to break anything on you. And of course, please report any issues or other feedback.


The current changes/fixes versus the most recent trunk are:

- [fix] Show guild tags again in the spawnlist and map tooltips. This requires a couple of additional opcodes that are already added to the dev branch and current as of the 02/17 patch

- [fix] Show held/worn equipment again in the spawn list

- [enhancement] If the network interface that's configured/requested doesn't exist, present a list of options to choose from instead of failing to start

- [fix] Fix a couple of issues with message display/formatting

[fransick]

Going to push the spell list changes up to your devel branch for you to try. Updated the struct and opcode and it seems to be working albeit inconsistently for me at the moment.

[fransick]

Committed a bunch of less important opcodes and fixed a few structs associated with those opcodes in the devel branch. Was bored and thinking I needed to get back into the swing of things re: opcodes and structs. You may see more spam in the console as a result. Enjoy!

[cn187]

The current batch of changes on the dev branch have been merged into trunk as of 6.1.0.0.

I'm not sure what next bits I'll be working on, but it will probably be a least a couple of weeks before I seriously start anything due to IRL workload.

[BlueAdept]

No problem. SEQ has been around for 21+ years (Hey, it is of legal drinking age!), it can endure a couple weeks off.

[cn187]

I forgot to mention it when I did it (in late April) but I added some Qt5-related fixes to my dev branch. With those fixes, I'm able to compile against the Qt5 version that ships with Debian Buster.

If you have both Qt4 and Qt5 installed, it will prefer Qt4. You should be able to force Qt5 with configure flags, but I haven't tried it on a system with both installed.

I haven't done much testing (I haven't had time to play lately), so I can't vouch for how well anything works under Qt5, but if someone wants to poke at it and post bugs/issues, be my guest.

Aside from those Qt5 fixes, the dev branch is currently up to date with trunk. Hopefully I can dedicate some time to making more progress soon.

[BlueAdept]

Cool. I appreciate the effort. Moving to qt4 was a huge deal! It is still widely available so the upgrade to qt5 will be when ever you get the time.

[cn187]

Notable recent updates to the cn187_devel branch:



Change fillSpawnStruct to loop through posData (based on struct size)
rather than hardcoding the fill of each element. This will allow the size
of posData to change without needing to adjust fillSpawnStruct.




Add safety check for formatted message arg sizes.
This works around a crash (due to possible packet changes) until a
proper fix can be found.




Check sessionId before processing disconnect packet

Random UDP traffic on the LAN can cause unexpectes session disconnects if
the packet payload happens to start with the SessionDisconnect net
opcode.

By checking the sessionId in the disconnect packet against the current
sessionId, we can ignore packets that look like disconnect packets but
really aren't.

This also means that users who have enabled "session tracking" for the
sole purpose of reducing random disconnects may be able to disable it if
they don't otherwise need it.




Add the ability to load multi-layer maps (e.g, SOE, Brewall's, Good's).

* On zoning, SEQ will automatically load the base zone map as well as maps 1-3
of the same name.

* The layer control on the map bottom control pane allows you to select
which layers you want to see/hide.

* Manual load/import of maps will now allow the selection of multiple files

* Saving maps will save all layers as their individual filenames

* The map edit menu now contains an "Edit Layer" setting, allowing you
choose which layer will be modified when doing map edits.


I expect the first two will go to trunk pretty quickly (along with some other minor things). Probably by the next patch to live.

The third, I'm debating whether or not we need a toggle to enable/disable this (right now the change is implemented globally). I don't think it will have any undesirable effects for single-boxers, multi-boxers that play on different PCs, or anyone that has "session tracking" enabled - it should just result in fewer random session disconnects.

However, I think it will be a behavior change that might affect multi-boxers that play multiple toons on the same PC and don't have "session tracking" enabled. But I have no clue how common this combination is.
I haven't tested how it behaves in that scenario, so it may all be moot, but in my head, I think that instead of any toon zoning causing SEQ to zone, the toon that SEQ is tracking (as Player) will have to zone in order to zone SEQ (but from there, it would be the usual race to see which toon SEQ latches onto). Part of me thinks no one will care. The other part of me remembers https://xkcd.com/1172/ so I'd be really interested to know if anyone would be bothered by this change.

The fourth, I binge-implemented this past weekend. I've done a fair amount of manual testing, but haven't put in any serious play time yet. I'd like this to sit in the dev branch for a while and have people play with it in case there are bugs/issues I missed. I think one gripe will be the map colors. I think the conversion script changed the map colors to work better with SEQ's color scheme, so trying to use the maps directly might mean the colors are hard to see. Maybe a short term solution is to rework the map conversion script to just do colors and not squash the map layers together. Longer term, it might be nice to have the map color replacement happen automatically inside SEQ (maybe as part of the color-handling changes that will be needed for supporting different color schemes/themes).

[cn187]

it might be nice to have the map color replacement happen automatically inside SEQ

I added the color replacement for the SOE-formatted maps to the map load code. So pending any bug reports or other issues, the mapconvert script may no longer be necessary since SEQ is able to handle both multi-layer maps and color conversion. There are still color improvements that will likely need to be done as part of the theme support, but this is a first step, at least.