Bit Patterns

[Fate-JH]

This update originated in a different yet-incomplete update working on implementing the vanu benefit modules from the caverns but is being deployed as an independent update as it has nothing difrectly to do with the vanu modules. The purpose is to massage incongurences between hard-worked analysis of packet formats and newly-discovered static analysis on transcoder formats.

Detailed
Unlike other aspects of the base game, the caverns were near completely inaccessible to players during the last days of the server where the majority of our network data scraping was performed. The weapons and vehicles from the cavern expansions could be accessed and toyed with through the virtual training ranges and correct object creation packets could be extracted from those regions. Unlike them, the vanu modules are only ever spawned in the caverns and then must have to have been transported into the larger part of the game world to be used, meaning that their object creation packets could never be captured with basic cavern blocked. All other object creation packets were exhaustedly recovered through packet capture tools and painstakingly anaylzed at the bit level by hand, meaning that an object whose packet was never sampled nearly impossible to replicate. Many game objects can technically be spawned by selecting an object spawning transcoder and giving it the correct object id; but, without the correct payload, the resulting object will probably not function correctly.

Analysis on the client decompile ultimately traced object creation pathways and produced potential transcoder patterns for different kinds of object the game can spawn. This includes the vanu modules and it's the primary reason why the vanu module update in the works can even be properly feasible in the first place. The description of the vanu module transcoder includes the transcoder for the capture flag - the lattice logic unit - a semi-common entity that can be spawned at certain major facilities and must be transported to other facilities for the purposes of base capture. Part of this relationship between capture flags and the vanu modules were always anticipated mechanically but extending to the packet level is very meaningful. The main issue, then, is the common field data - a series of flags and recurring information in object creation packets for almost all entities that can be spawned.

Since having relied upon manual analysis for determing packets from the start, this region of common data has been split. Both have had a defined structure that includes the same fields serving the same purposes but the main difference is the unexpected bit of data that interjects somewhere in the pattern and causes important fields to have to be shoved around. Determining the location of this extra bit, and the reasons when and what it exists, was too had to figure out. Failure to respect this extra bit results in the transcoder for the given entity, and anything that might be streamed along with that entity, i.e., as in game obejcts in an inventory, breaking down and all objects after it being unrepresented. The capture flag, too, uses the common field region, but, since capture flag does not go into an inventory in a conventional, the precise length of the common field versus the pure object transcoder payload could never be properly determined. On the positive side, the previously-mentioned object creation pathways of the decomile also shed light onto the structure of both the capture flag and common data fields.

Outside of that, and in general, this is one of those updates that looks massive but exposure reveals not much has changed at all and everything should still work the exact same way upon playing normally.

Features
Prefab
Originally, these overloaded constructors were for the assistance of vehicle creation at the pad, but it was never properly utilized. The real load incurred by the existence of these prefabrication defaults was the existence of excessive and obtuse overloaded constructors on a variety of other classes. There's no reason to keep any of these functions around.

CommonFieldData
The pattern of common flags for most the object create transcoders of many game objects, occasionally the only pattern for the object. The known parameters of the common flags for a majority of the game's objects follow:
faction - 2u - Usual faction affinity datatype, but it does not always need to be set to a specific valid faction.
bops - 1u - Not all objects have a form or texture that reflects allegiance to the "black ops" mercenary splinter group, but, if they do have such a texture different from their common form or their faction-aligned forms, this flag activates it. This is technically different from if the faction is reported as being of NEUTRAL allegiance.
alternate - 1u - A better description of this flag might be "destroyed" but not all models have a "destroyed" form. Vehicles become wreckages, player characters become corpses (presents or pastries), deployables become damaged husks, etc., but it's believed this flag also serves purposes in representing special animations such as zip line halos and being locked into the cavern core beam.
v1 - 1u - A flag of unknown purpose though most frequently encountered in situations of one entity being contained within another or in a state of being possessed by another. It's not consistent across the sampled entities so mere packet analysis is not enough to determine the reason. In general, it should be true for you, but false for others or if shared.
jammered - 1u - On entities that can report being unable to operate when subject to an effect - called being jammered, after the common grenades that extend the effect - this flag can be set. The result does often remove the game object's ability to function normally, and occasionally generates a sound audible to the game world.
guid - 16u - The primary purpose of this field defines the owner of the game object, when the object can be considered "owned" by some other game entity, usually a player character.
In between v1 and jammered is a two-part field that was uncovered in the decompile analysis. The field is a combination of two values gates behind two otherwise unknown client-originating decision statements. One field is 1u and is almost always false, The other field is 16u and is gated behind both the first and a second decision statement. There are no packets that utilize this second field so it's purpose is unknown.
In between this unknown field, mixed in with jammered somewhere, but before guid is another optional 16u field, but one that is fortunately flagged by the 1u bit the precedes it. The only documented use of this field is repeating the router's GUID in the data of the router's internal telepad component. Originally, the single undocumented bit that appears for some game entity transcoders was inserted as a hard 1u in this region somewhere and the common data transcoder had two forms split between one having this bit and one not having this bit.
Under this rewrite, the strange single bit has been removed, merging the two forms of the pattern again, and the new position of the mysterious occasional bit is whether or not the other unknown region of the pattern - the unused 16u and the 1u - is represented. The trigger conditions for these two parameters are unknown, however, as has been mentioned, so once again the transcoder split is retained, either always on or always off. Purity of structure, but not clarity.

ToolPatternData (DetailedWeaponData, DetailedContsructionToolData, DetailedREKData, DetailedCommandDetonaterData)
A transcoder extraction pattern, utilized in various equipments transcoders like typical weaponry and the REK. This was known prior to the most recent analysis, at least a far back as the original object create analysis, but the parameter distribution was not well-defined at the time and it did not get implemented as a centralized transcoder. Except for the enumerated encoded data regarding ammunition types allocated to the piece of equipment, however, the purpose of most fields in this pattern is still not well understood.

RemoteProjectileData, LittleBuddyProjectileData (defunct)
The OICW was a late addition to the weapon transcoders due to its unique projectile mechanics. A primary projectile from the weapon travels a set distance, then explodes and spawns a few lesser projectiles that perform the actual damage accorded to the weapon. These secondary projectiles are known internally as "lil buddies". When first examining the object create packet data for the projectiles, it was separated from other object create projectile transcoders because it activated different fields and neglected others formerly considered common to projectiles. Through proper analysis of the base projectile transcoder, the data normally applied to the little buddy transcoder was recalculated for different fields and has been collapsed into the sme base projectile encoder. The payloads were always the same bit size.

TelepadDeployableData
Fun story about this transcoder. Based on the sample data from the packet captures, the converter was assembled to always utilize the same data in the previous fields based on the packet captures for this game object (which are few, understandably, as routers are not something that can be spawned normally). The resulting parameter data doesn't really make any sense, but it also doesn't break anything to use it. The new analysis on game object transcoder suggested a single 1u separating a 16u which fits within the bounds of the payload, and all of a sudden the data extracted from the payload resembles something that made sense!

CharacterConverter
Felt a need to separate this from its original AvatarConverter class so to make its functions more available to other converter classes that also manipulate character data into packets.

CaptureFlagData
Originally, I did not intend to change this transcoder too much as, even if it would be required to get it working correctly for vanu modules, I have already adjusted it within the corresponding branch and that will create merge conflicts for me later. Instead, I found it too compelling to not correct the inaccuracies with the transcoder here. Most of it was correct. I don't have an excuse for why I for 8u (unused) + 8u (defined) couldn't turn into uint16L, but I suppose being stupid sometimes isn't as bad as it sounds. At the very least, spawning a capture flag with this improper arrangement never blew up in my face.

Caveats
Originally, I intended to perform maintenance work on the transcoder for vehicles - VehicleData - but the new analysis provided for a transcoder pattern that wasn't very useful to the packet capture sample data. Additionally, regions of the decompile revealed bit extraction patterns that were similar to the previous manually-composed vehicle transcoder pattern; but, at the same time, did not meet the expectations of that pattern in either length or composition and thus were useless to parse the data as well. Ultimately, no work was performed on the vehicle transcoder beyond what was requried for adjusting common field data and the original manually-composed transcoder is being retained for the time being.

Adenda_

1. I goofed by changing the directory structure involving game packet transcoders and the object create payload transcoders. I wanted to separate the game packet transcoders from the object transcoders by not having the former always be visible when the latter is, but I moved one group too soon and then moved that same group of files back in between different commits. That resulted in an expansive turmoil that changed imports throughout the project. The number of files changed for this pull request is 651 even though not much has actually been changed in most of the files.
2. All game object converter classes have been ... converted ... into objects that no longer need to be instantiated. I do not remember why they were designed as instance objects in the first place. If it ever becomes necessary to inject and retain information local to a specific converter, that converter can always go back to be a instance class.
3. For the \objects\ directory, most if not all instance of PlanetSideGUID(0) have been exchanged for Default.GUID0.
4. There's no rule that you can not name your pull request the same as the branch it pulls its changes from.

[Fate-JH]

It has come to my attention that almost every facility turret in this instance has spectacularly failed with both a lack of owner and a missing GUID issue.
Locally, turret errors do occur but barely more than a few of them suffer like this.
If this continues, at least the point of error can be easily determined during issue debugging. It wasn't common enough before.

[Fate-JH]

I know attaching a potential solution to a major issue experienced in many other parts of the game is probably not good design BUT this is also the best place to have identify it and to present it for testing. The server, upon each restart, was suffering from a few facility turrets failing during startup before this update. Reliably. Literally all of them for this test server.

As of this update, however, the test server started correctly on the first try.

[Dethdeath]

Only thing I've seen so far is gmtoggle on/off makes the player face the ground.

[Fate-JH]

Updates:
Features
DriveState
Converted from standard enumerated value to our alternate library enumeration type. Additionally, added some new specific values.

VehiclePatternData (VehicleData, OrbitalShuttleData,DroppodData)
Despite the incongruence between the provided analyses of a decompile provided, and despite its focus on object create patterns not being thorough enough, the vehicle transcoder that is being used as a baseline had a portion of its code separated. This "common vehicle pattern data" was applied to the transcoders for the orbital shuttle and the droppod, and works by playing on actual similarities between same-length transcoding patterns. Additionally, one transcoder for the orbital shuttle can be replaced by a straightforward implementation of the vehicle transcoder. This has, however, produced peculiar value discoveries such as the custom DriveStates for both vehicles, and an oddity that says the shuttle has 31/255 health which is dubious but also well within the margins of the pattern. Both will continue to be tested under the heading of a "non-standard vehicle".

Caveats
The original listed entry can be either amended or removed as per the inclusions to "features" listed before this.

[Dethdeath]

Only thing I've seen so far is gmtoggle on/off makes the player face the ground.

Turns out this is actually caused by an earlier build.

[Fate-JH]

Turns out this is actually caused by an earlier build.

So it should work just fine once we merge? That was the last thing I was worrying over.

[Dethdeath]

Probably, I've tested a bunch of random things but the real test will have to be during a fight on our regular server.