Add radial bounds to PositionLimits#939
Closed
lgulich wants to merge 65 commits into
Closed
Conversation
Import the configclass decorator from its defining module so lazy package imports cannot shadow it. Defer physics-preset imports until SimulationApp startup to preserve config class identity during collection. Handle the solver, stage, and camera API shapes used by the public Isaac Lab wheel while retaining compatibility with Arena's Docker submodule. Skip the live endpoint test when its credential is absent. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Move package metadata to PEP 621 and add an Arena-owned uv lock for Python 3.12, the public Isaac Lab 3.0 beta wheel, Isaac Sim 6, and the matching CUDA PyTorch stack. Keep simulator dependencies in a default PEP 735 group so Docker continues using its pinned Isaac Lab submodule. Document the native developer workflow and add a lightweight CI guard for lock consistency, editable package roots, and runtime resources. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Use strict first-index resolution and route the sole incompatible NVIDIA/PyPI name collision explicitly to PyPI. Keep the locked package graph unchanged while making future name collisions fail closed. Clarify the editable-package check and ignore auxiliary uv environments. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
- Default --output_base_dir to ./outputs so the runners work natively and in Docker without flags (gitignored); document /eval as the override. - Lock resolution to Linux x86_64 (tool.uv.environments); drop the per-platform torch markers and the cu130 index. - Move the uv lock/editable validation out of the pre-commit job into its own uv_package CI job. - Align native and Docker verification docs (optional test phases in both); reword the uv docs positively and drop the redundant note. - Annotate the two Isaac Lab version shims (get_local_poses ProxyArray, get_current_stage location) for removal on version convergence. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
The subprocess-spawning tests hard-coded the Docker submodule interpreter (submodules/IsaacLab/_isaac_sim/python.sh), which does not exist in the native uv environment, so every with_subprocess test failed at spawn with FileNotFoundError before any Isaac Lab code ran. Fall back to the current interpreter (sys.executable = the uv venv python) when the Docker python.sh is absent; Docker keeps using python.sh unchanged. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
With the interpreter fix the native uv env runs the with_subprocess phase, but some tests cannot pass outside Docker for environmental reasons. Mark them with shared skipif markers (active only when the Docker python.sh is absent): - requires_docker_assets: tests that load assets not yet promoted to the public Nucleus (maple_table.usda 404s natively). - requires_docker_isaaclab: tests hitting API differences between the Docker Isaac Lab build and the public 3.0.0b2 wheel (ArticulationCfg in the RL training env; contact-view filter_count in mimic data generation). Remove once the Isaac Lab versions converge. Native result: 9 passed, 11 skipped (was 9 passed, 11 failed). Docker runs the full set unchanged. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
A floor grocery-packing scene (LIBERO "object" suite) for the cap project: ground-mounted Franka Panda + basket + 6 HOPE groceries (alphabet soup, tomato sauce, milk, salad dressing, cream cheese, butter). Object placement is declarative via the relation solver -- On an invisible floor surface, bounded to the Franka reach box (PositionLimits), with per-reset jitter (RandomAroundSolution) -- instead of hardcoded poses. Registered as "libero_object_packing"; runs with the stock ILA scripts (e.g. teleop.py --viz kit). Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
- Mark the basket as IsAnchor so the relation solver treats it as a fixed obstacle (no-overlap loss + validation) rather than ignoring it; groceries no longer spawn inter-penetrating the bin and getting ejected by physics. - Clean the basket quaternion to a pure 90deg yaw (opening still up) so the anchor bounding-box path, which only accepts yaw rotations, accepts it. - Add isaaclab_arena/scripts/view_environment.py: a live viewer that builds a registered env, resolves placement, and idle-steps it for visual inspection. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
- libero env: --control joint_pos selects the franka_joint_pos embodiment with an absolute JointPositionActionCfg (scale=1.0, use_default_offset=False) so a commanded q lands at q; default stays relative IK (zero action holds pose, for the viewer). Use HIGH_PD gains for joint-pos so the arm tracks the target instead of sagging ~0.25 rad at joints 2/4 under gravity. - Add cap_bridge M1 (env/joints/wrist-cam smoke) and M2 (absolute joint-command round-trip) harnesses validating the milestone gates from Isaac-cap docs/ila_side_design.md. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
- Mirrors the Isaac-cap stub_ila_client framing (4-byte length + msgpack_numpy) but drives the real env: send obs (raw joint_pos + gripper fraction + blank robot0_robotview frame) -> recv GaP's absolute joint+gripper target -> apply via the absolute joint-position term -> env.step. - Streams an observation before reading the first action so GaP reset() unblocks; maps GaP's continuous gripper fraction to the binary finger term at a 0.5 threshold. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
GaP reset() waits only ~60s for the first RGB frame, but booting Isaac Sim takes minutes. Boot the env first, then retry-connect to the server, so the operator can start GaP after the env is ready and GaP's clock starts when the client is about to stream. M3 motion smoke passes end-to-end: observe->move_to_joints completed->done(success). Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
- Add a fixed exterior rgb+depth camera (top-down over the table) and depth to the wrist camera via a libero-local FrankaCameraCfg subclass, injected only when --enable_cameras (core embodiment untouched). - Add the R5 frame-check: build pose_mat = T_world_base^-1 @ T_world_cam(quat_w_ros) (OpenCV, no flip) with Isaac frame utils, then replicate GaP's exact back-projection and confirm known objects reconstruct to GT base-frame position. PASS: 0.000cm frame error over all 6 objects. - Adversarially verified the pose_mat convention (3 independent derivations) before implementing. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
- m4_bridge_client streams the real exterior camera (rgb + top-level depth_data + intrinsics + pose_mat) so GaP's DINO/SAM3/VLM perceive subgraph runs on real ILA renders. Depth goes under depth_data (what GaP _convert_observation reads), NOT images.depth. - Aim the exterior camera agentview-style (top-down fails open-vocab recognition) and derive a deterministic pose_mat = T_base^-1 @ [R_gl @ diag(1,-1,-1) | eye] from create_rotation_matrix_from_view; quat_w_ros is stale/ambiguous right after set_world_poses_from_view. - Gate: alphabet soup can OBB center 0.91cm from ground truth (base frame, <1cm). Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
…erification The single-object pick-place (M5a) over the bridge places the soup can in the basket: GT-verified final can pose 4.3cm from basket center, z in range. Full cuRobo grasp + transport ran on the correct target via the editable pick_place graph (object_name 'the alphabet soup can'). Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
…dge (M5b) - m4_bridge_client: stream the exterior agentview RGB to an mp4 (--record_video, lazy imageio import to keep it off the pre-SimulationApp import path); log GT IN_BASKET count for all objects each tick. - Add cap_bridge/README.md: wire contract (incl. top-level depth_data + R5 pose_mat = T_world_base^-1 @ T_world_cam, OpenCV, distance_to_image_plane), scene additions, launch, and M1-M5 status. - M5b: full packing loop packs 5/6 objects (cream_cheese skipped as too-flat), terminates cleanly, video recorded; all milestones GT-verified. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
…port - --camera_every N: render+refresh the exterior camera every Nth tick and reuse the cached frame between (the camera updates lazily on access, so skipping the read skips the render). This is the real per-step speed lever: GaP only reads the camera at perceive nodes, but the obs path was rendering the 800x512 exterior cam every tick. Video is already opt-in (default off). - Make imageio a lazy import (only when --record_video) to keep it off the pre-SimulationApp path. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
- Per-step timing split + periodic + final TIMING print: obs(render+send) | host_wait(recv=GaP) | env_step(sim). Periodic print survives an end-of-run hang; recv socket timeout (60s) makes the client exit cleanly when GaP finishes (it leaves the socket open after done). - Profile (reverted settles, 5/6 packed, 1130s): env_step 91% (per-step ~39ms, render-heavy), host_wait 3%, obs 6%; phase wall = motion-exec 65% / perception 26% / cuRobo 6% / settle 2%. Wall is sim-step- bound, not host-compute-bound; cuRobo planning is negligible. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
…e lever) - m6_render_probe: measure mean env.step with/without exterior-cam read. Result: render is in env.step (access 33.4ms ~= no_access 32.8ms), so skipping the obs READ does NOT skip the render; physics-only floor is 15ms -> render ~= 18ms/step. - libero_cameras: ILA_CAM_UPDATE_PERIOD env var sets the perception cameras' update_period (default 0.0 = every step). 0.08 (~every 4 steps) -> 23.5ms/step, ~30% per-step; capped by sim render_interval=2. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Skip env.step while the commanded action is unchanged AND the arm is settled (>_IDLE_WINDOW steps), so GaP's host-compute idles (perception) don't burn idle physics steps; the in-window steps render the settled scene so perception still gets a fresh frame. Default on (--no-idle_skip to disable). Mechanically halves env.step count during idle; report env_steps vs skipped. NOTE: the idle period is largely GaP-bound (perception runs async and reads the camera during the hold), so this is mainly a CPU/GPU saving -- the wall is dominated by perception-cycle COUNT variance, not idle stepping. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
GaP closes the socket after 'done'; the next _send raised BrokenPipeError (it was outside the try), crashing the client before the TIMING summary. Wrap _send in the same break-on-OSError path as _recv. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
…lt 2) Perf probe finding: render_interval is NOT the binding render-cadence lever (2->8->16 didn't move per-step); the camera update_period is. Kept as an opt-in knob (default 2 = unchanged). Per-step floors at ~20.6ms (steady probe) via update_period; the 15ms physics floor isn't reachable via cadence. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
…eries) - Add a visible static table: the procedural_table instance becomes a thick kinematic cuboid (floor->0.40m, 1.2x1.6) via a per-instance spawn override, doubling as the relation-solver On()/anchor surface (top at z=0.40). Shared asset cfg untouched. - Mount the Franka on the table: base -> (-0.20,0,0.40). Raise basket -> z=0.40; groceries On() rest on the table top. Raising base+assets by the same H preserves base->object geometry, so reachability and top-down grasps are unchanged. - Re-aim cameras for the new heights: exterior cfg cam z 1.2->1.6 (frame-check), client agentview _CAM_EYE/_CAM_TARGET +0.40. - Validated on the tabletop: M4 frame-check PASS (R5 pose_mat 0.000cm over all 6 objects, objects rest at z~0.44, table visible) and a full pick_place perceive->grasp->transport->place (object placed in basket). Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
…tive import fix LiberoPackingTask wires objects_in_proximity(target, basket) -> check_success -> 'success' term + SuccessRateMetric (default 'none' keeps NoTask so viewer/m4 runs are unaffected). Exclude libero_cameras from eager auto-import so eval_runner's module-top import works under a native uv venv (was the real cause of eval being Docker-only). Enables GaP-as-PolicyBase eval via eval_runner (verified end-to-end). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
…ero_object_packing DroidAbsoluteJointPositionEmbodiment (own USD/action/PD, 13-DOF, Robotiq) via --control droid_joint_pos, with LiberoDroidPerceptionCameraCfg adding the exterior_cam (rgb+depth) the bridge reads (base link panda_link0 => pose_mat unchanged vs the panda scene). Stand USD 404 on public Nucleus worked around with an invisible cuboid. Enables GaP eval on the DROID embodiment alongside the panda franka_joint_pos path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
…acking - terminations: add resting_in_bin (footprint + below-rim z + settled velocity) and gripper_open release-guard predicates - libero_object_packing: LiberoMultiPackingTask (check_success SuccessMode.ALL over N resting_in_bin + gripper_open); default 3 graspable groceries; pooled placement with 6cm inter-object clearance; reach box tightened to Panda top-down reach; episode_length_s=150*N+150 - relation_solver_interface/arena_env_builder: optional clearance_m thread (env opts in via placement_clearance_m); backward-compatible Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
…ROID pending, env refs) Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
…d lock pin (no combined CAP branch) Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
…r list args - job_manager: expand list/tuple arena_env_args into post-flag CLI tokens (--objects a b ...) via a new _render_cli_arg helper, instead of str(list); add test_job_convert_args_dict_expands_list_values. - libero_object_packing: new --eval_task stock_sort backed by the STOCK SortMultiObjectTask (per-object object_on_destination under SuccessMode.ALL, N objects from --objects); remove the hardcoded 2-object scene workaround and the dead custom two-pick factory. - Set task_description on SortMultiObjectTask so get_language_instruction() is populated for the GaP remote policy. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
…riation indices Scene-smoke-only: no GaP E2E success is claimed yet (CAP lifecycle + obs metadata pending). - pick_and_place_maple_table: new opt-in --gap_profile (default off → stock + VLA jobs unchanged) that attaches a fixed exterior RGB-D agentview camera, registers its camera_extrinsics_exterior_cam variation, and constrains object placement to the arm reach box. Fails early unless --enable_cameras + a DROID embodiment. --episode_length_s configurable (stock default 20.0). - --use_staging_assets (opt-in, fail-closed, logged): load the unpromoted PR #786 Maple-table + DROID-stand assets (404 on production, 200 on the Isaac staging bucket) via a targeted host swap. Override is instance-local — background via a dynamic subclass, stand via deepcopy — so it cannot leak into later non-staging jobs/rebuilds. Asserts (not silently skips) if a stand usd_path cannot be rewritten. - maple_cameras.py: MapleDroidPerceptionCameraCfg static agentview exterior_cam for the live-pose adapter contract; quaternion uses this Isaac Lab's xyzw order. - camera_extrinsics_variation: pass a warp int32 index array (not env_ids.tolist()) to set_local_poses; the UsdFrameView backend resolves indices via .numpy(), so a list crashed every camera extrinsics variation. - job_manager: add "seed" to the global priority args so it precedes the env subcommand. - Tracked scene-smoke job eval_jobs_configs/maple_gap_scene_smoke_jobs_config.json (zero_action, gap_profile, staging, exterior_cam variation, explicit seed). - Tests: warp-index regression, staging no-leak, agentview xyzw look-at, seed-ordering. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
…file Under --gap_profile, raise num_rerenders_on_reset to >=5 in the viewport cfg callback so reset-time placement and camera-extrinsics variations are rendered before the first GaP observation. Opt-in only; stock single-object / non-gap_profile behavior is unchanged. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Preserve the camera index representation supported by both the public-wheel and Docker-pinned Isaac Lab APIs. Use the direct configclass decorator import in CAP-added modules and clean the legacy bridge scripts so the rebased tree passes pre-commit. Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
isaac_cap is overlaid into this env editable/--no-deps; its wire-format deps previously lived only in the venv (0 hits in uv.lock), so any `uv sync` silently dropped msgpack-numpy and broke the GaP bridge at import. Pin them in Arena's own lock; the overlay stays dependency-free. Version matches isaac_cap's ==0.4.8 pin and the GaP-side lock. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Signed-off-by: Rafael Cathomen <rcathomen@nvidia.com>
Signed-off-by: Lionel Gulich <lgulich@nvidia.com>
Signed-off-by: Lionel Gulich <lgulich@nvidia.com>
Signed-off-by: Lionel Gulich <lgulich@nvidia.com>
Signed-off-by: Lionel Gulich <lgulich@nvidia.com>
Signed-off-by: Lionel Gulich <lgulich@nvidia.com>
Signed-off-by: Lionel Gulich <lgulich@nvidia.com>
Signed-off-by: Lionel Gulich <lgulich@nvidia.com>
Signed-off-by: Lionel Gulich <lgulich@nvidia.com>
Collaborator
Author
|
Superseded by #940, which is rebased on current main and contains only the generic radial PositionLimits feature. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PositionLimitsrelation with optional world-XY disk/annulus bounds.relation_loss_weightcompatibility and add a deterministic exact-center lower-radius gradient path.Motivation
Rectangular bounds cannot express radial placement regions without approximating them with a box. This adds disk/annulus constraints while keeping existing axis limits composable.
Validation
pre-commiton changed filescompileall/ syntax checksFocused Isaac Sim pytest remains pending: this checkout has no pinned local environment or running Arena Docker container. No bootstrap or image pull was performed.