Skip to content

[Feature Request] Light-weight inference pipeline tailored for driving standard FBX/Humanoid avatars (Skip EHM mesh reconstruction) #13

Description

@SABBA-HOSTUMEX

Hi PEAR Authors,

First of all, thank you for this incredible work! The pixel-aligned accuracy and the low-latency design of PEAR are absolutely amazing.

I am currently working on a real-time motion capture project using PEAR to drive standard 3D avatars (e.g., standard FBX models, VRM, or Unity Humanoid hierarchy) in a game engine via UDP.

Since my goal is strictly to animate an existing production-ready FBX character, I only need the joint rotation data (such as the Axis-Angle representations for Body, Left Hand, and Right Hand). I do not need the dense 3D mesh vertices, nor do I need to run the full ehm() forward kinematics / linear blend skinning (LBS) on the Python side, which introduces significant computational overhead (pushing my RTX 3090 "3D" engine usage to 95% at times).

Is it possible to provide a streamlined, inference-only pipeline that:

  1. Only extracts the raw skeletal structure and local joint rotations (e.g., Zone A, B, and C).
  2. Completely bypasses the EHM mesh reconstruction and the generation of the 358-float vector (skipping Zone D, E, F).
  3. Provides a direct mapping concept from PEAR's predicted joint rotations to standard skeletal hierarchies?

Having a lightweight pipeline dedicated solely to extracting rotation data would make PEAR the ultimate tool for real-time XR and game development.

Thank you again for your time and contribution to the community!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions