Integration into existing XR components

[Rectus]

I just released a simple Vive tracker plugin (https://github.com/Rectus/UE4OpenXRViveTrackerPlugin) that uses the existing motion controller component in the XR system to provide the tracker pose. It also works out-of-the-box with 4.27

I'm wondering if it might be a better idea for the official plugin to use the existing abstracted component already present in the engine, instead of having a separate component class for the tracker. The motion controller component is already designed to handle arbitrary motion sources. There are currently a few issues with it, like the GetMotionControllerData function only accepting legacy hand enums, but the basic functionality seems to all work. The only thing I can think of that would be unavailable is the unique tracker ID.

[1runeberg]

Hi @Rectus - thanks for sharing!

It was one of the design considerations to use the MC, but as you've encountered, there are certain features that devs might expect of a MC but suddenly unavailable so we couldn't make a feature-complete MC for the tracker. This is a similar issue with the current SteamvVR Input plugin, we managed to add custom motion sources but an engine CL was rejected as Epic is working on a more thorough fix to properly allow custom motion sources in (e.g. visualizations break):

• Is there something specific with the MC that you need that's not available with the custom Tracker/Scene Component?
• Where does the GripID come from?
• Does this work with multiple trackers enabled?

Also, I haven't tried it yet, but it seems that actions are created for each MC that's in the user project and poses are also checked for each role in every MC?

Unfortunately there is no disconnected event in the spec, but I think you should be able to detect tracker disconnection in your existing event check if you're caching all active trackers or checking the tracking flag for the pose action.

[Rectus]

There isn't really anything that the MC can do that a custom component can't. The main idea is just to fit it into the base XR interface. The motion source system seems to be designed with this in mind, and it would be nice to have it in there already in case it rolled in to it in the future.

Also that it works with an unmodified 4.27 engine seems like a big plus (although the changes in your pull request would be very welcome).

The GripDeviceId is assigned by the OpenXRHMD in FOpenXRHMD::AddActionDevice(). I had to add a delay to the assignment, since the motion controllers themselves are hardcoded to to use the first IDs.

Unfortunately I only have one tracker to test with. Adding multiple components and toggling the tracker role in the SteamVR settings makes it update the correct component on the fly.

It creates a pose action for each role on the spec, and lets FOpenXRHMD::UpdateDeviceLocations() handle the updating, so it does attempt to retrieve poses for all possible trackers. It might be possible to add in devices for each role after they have been connected though, I'll have to test it out.

If I'm reading it right, the spec for the tracker extension states that the runtime must send a XR_TYPE_EVENT_DATA_INTERACTION_PROFILE_CHANGED event when a bound tracker is connected or disconnected. Testing it now, the runtime only seems to send the event on connecting. The spec for xrGetCurrentInteractionProfile() is unclear on what should be returned on disconnected devices, but returning an XR_NULL_PATH would be logical.

Thanks for the insight!

[1runeberg]

Re: Unmodified 4.27 - I'll double check with Epic if the changes for the OpenXR plugin won't come in for 4.27 (which I stil lhope it does since dpad can be implemented with the same patch), then AddActionDevice would be a great alternative for users not making the jump quite yet. I just need to do some due diligence on any side effects. But looking at the code it does seem that deviceid here is just the id from the openxr plugin and not a legacy device id. Thanks again for sharing, if we incorporate, will put a shoutout to you on the commit :)

Re: Multiple - It just seemed the way the actions are generated and used they might trample each other, but could be wrong and should be an easy fix. prolly for perf reason be good to keep track of only the active roles.

Re: disconnections - ah yeah true, will double check but thought i tested that while implementing the spec in the runtime. internally i keep track of the active trackers and discard disconnected ones. it might've regressed somewhere along the way. thanks for the report.

[Rectus]

Testing with only adding active trackers, and there are issues that prevent it from working. The UE4 OpenXR plugin automatically adds a new action space to the action when a tracked device is added. This works properly at session initialization, but not afterwards. The spec seems to be unclear of is adding new action spaces after attaching the action sets is allowed, but doing it with SteamVR seems to result in a action space that never becomes locatable.

There doesn't seem to be any way of enumerating the connected trackers until after the session is active, so I can't find any way to pick which trackers to add.

The xrEnumerateViveTrackerPathsHTCX function only works after the session has been initialized and in focus. The spec doesn't mention when it should be functional, and it only takes the instance handle. I think it should be functional outside of sessions. It would help with determining which trackers are active at launch.

Also, I noticed a few more issues:
The XR_TYPE_EVENT_DATA_VIVE_TRACKER_CONNECTED_HTCX event doesn't fire when a tracker role changes.
xrGetCurrentInteractionProfile works now for all roles, except handheld_object.

[1runeberg]

Hi Rectus -

• Actionset attachment uses UE4's standard workflow if you're using the appropriate engine patch - can you clarify exactly what you are trying to achieve in your app?
• The trackers are input devices and the OpenXR spec requires that input can only be active if the session is in the focused state.

[Rectus]

I was attempting to get my motion controller based version of the plugin to only pass the active devices to be tracked by the HMD class. I guess this approach won't work with how the stock OpenXR plugin is set up.

Thanks for clarifying about the spec, that makes sense.