Add RoboLab (Isaac Lab DROID benchmark) behind the env-server#469
Open
vertix wants to merge 3 commits into
Open
Add RoboLab (Isaac Lab DROID benchmark) behind the env-server#469vertix wants to merge 3 commits into
vertix wants to merge 3 commits into
Conversation
…process launcher, `remote_franka_embodiment`
…rlay, e2e gates boundary mechanics
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Integrates NVIDIA RoboLab (Isaac Lab DROID benchmark, pinned to
NVLabs/RoboLab@7d45d74) as a remote eval environment behind positronic's env-server wire.WireCommandAdapter, the subprocess launcher,remote_franka_embodiment— and migrates libero and the env-server test fixture onto it.positronic/simulator/robolab/(env,adapter,launcher,validate,e2e,make_fixture) pluspositronic/cfg/eval/sim/robolab.pywith the pinned 120-task table and named configsbenchmark/visual/relational/procedural/banana_in_bowl/rubiks_cube_and_banana.Design decisions
joint_posis a passthrough (bit-identical to the RoboLab leaderboard stack),joint_vel→q_measured + dq,hold→q_measured, and Cartesian commands go through a standalone Isaac LabDifferentialIKController(DroidIKActionCfg parameters; commands and observations share theeef_frame= base_link ∘EEF_OFFSET_ROT (0.5, -0.5, 0.5, -0.5)wxyz).eval.seed(built directly, not viabuild_trials).done/successfollow RoboLab's frozen-env accounting (all_terminated+get_env_results), not raw term/trunc — their env discards ≤2-step "physics artifact" terminations, so raw term would score false successes.git lfs pulls assets, and patches upstream'sconstraint-dependencieswithcffi != 2.1.0. Upstream ships no lockfile, so fresh resolves can break on day-fresh releases, and--exclude-neweris impossible (pypi.nvidia.com publishes no upload timestamps). A full freeze (lockfile or baked image) is a parked TODO inlauncher.py.No CI story
Both verification gates need an RTX-class GPU and run on-box; nothing beyond the normal unit suite runs in GitHub CI.
Test plan
On-box verification (Nebius L40S, driver 570):
validate.py: ALL CHECKS PASSED — obs contract, grip extremes, bit-identical jointpos passthrough (0.0002 rad convergence), exactq + dqanchoring, eef↔base_link offset round-trip, 13/13 Cartesian IK cases ≤0.12 mm / 0.04° (tolerances 5 mm / 2°; including their known-divergenttranslate +xcase), cartesian_delta 0.06 mm.e2e.py: E2E REPLAY PASSED — exact-state reset drift 0.0115 rad, full 530-action socket replay clean. Replay task success is reported informationally only (0/1 here): the shipped demo recording is stale at the pinned commit — RoboLab's ownrun_recorded.pyalso fails it (0/1) on the same box — so demo replay is not a success oracle. The success oracle is the follow-up PR's policy differential (openpi DROID policy vs their leaderboard client).uv run --locked pytest --no-cov→ 606 passed, 7 skipped; ruff clean.