Skip to content

Add pinhole camera projection to reorder texture#11

Open
csparker247 wants to merge 7 commits into
developfrom
reorder-texture-camera-projection
Open

Add pinhole camera projection to reorder texture#11
csparker247 wants to merge 7 commits into
developfrom
reorder-texture-camera-projection

Conversation

@csparker247

@csparker247 csparker247 commented Jun 22, 2026

Copy link
Copy Markdown
Member

Summary

Adds a pinhole camera projection model to ReorderUnorganizedTexture, alongside the existing orthographic ("snapshot from above") sampling. Instead of casting parallel rays along the mesh's shortest axis, ProjectionMode::Camera casts perspective rays from a camera center, rendering the textured mesh as if photographed through a pinhole camera.

The camera is either:

  • supplied explicitly via a --camera-file (intrinsics fx/fy/cx/cy, output width/height, and a world-to-camera pose), or
  • auto-derived to frame the mesh — positioned along the shortest OBB axis, looking at the centroid, with intrinsics/output size chosen to preserve the input texture's pixel density.

Pipeline integration

Camera mode runs through the standard render graph rather than a separate code path. ReorderTextureNode gains two input ports:

  • projectionModesetProjectionMode()
  • projectionParamssetProjectionParams()

Both are serialized into the node cache (NLOHMANN_JSON_SERIALIZE_ENUM for the mode; to_json/from_json for the params struct, with the 4x4 pose flattened to 16 doubles). The rt_reorder_texture app parses the new options and wires them onto the node; the camera-file parsing lives in a dedicated static ParseCameraFile() helper that returns std::optional<ProjectionParams>.

Output handling

The output node branches on the requested file extension (works for both projection modes):

  • OBJ → writes the textured mesh (MeshWriteNode)
  • image (jpg/jpeg/png/tif/tiff) → writes just the reordered texture (WriteImageNode)

The -o option is renamed --output-mesh--output-file to reflect that it now accepts either.

Additional outputs

  • New per-pixel 3D position map (CV_32FC3) output / --position-map option and node port. Camera-mode positions are in the mesh's world frame; orthographic positions are in the realigned sampling frame. Pixels with no intersection are NaN.
  • Depth map semantics documented for both modes (orthographic: distance from the sampling plane; camera: optical-axis depth / camera-space Z).

Other

  • TextureDewarp: switch the dewarp sampling to OutputWidth at 8192 px for higher-resolution output.

Notes

  • Camera mode shares all outputs (texture, UV map, depth, position) with orthographic mode, so it flows through the same graph; the output writer is selected by extension as described above.
  • No textured-OBJ fixture exists in the repo, so this was verified by full build + CLI help; the ray/sampling math in create_texture_camera_() is exercised through the node rather than an end-to-end render test.

🤖 Generated with Claude Code

csparker247 and others added 7 commits June 22, 2026 16:47
Add a camera projection model to ReorderUnorganizedTexture alongside the
existing orthographic "snapshot from above" sampling. ProjectionMode::Camera
casts perspective rays from a pinhole camera center; the camera is either
supplied explicitly (intrinsics + world-to-camera pose) or auto-derived to
frame the mesh along its shortest OBB axis.

Camera mode is exposed through the standard render graph: ReorderTextureNode
gains projectionMode/projectionParams input ports (serialized for caching),
so no separate computation path is needed. The rt_reorder_texture app parses
the new --projection and --camera-file options and wires them onto the node;
the camera-file parser lives in a dedicated ParseCameraFile() helper.

Also adds a per-pixel 3D position map output (CV_32FC3) and documents the
depth-map semantics for both projection modes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Branch the reorder-texture output node on the requested extension: an OBJ
writes the textured mesh (MeshWriteNode); an image extension (jpg, png, tif)
writes just the reordered texture via WriteImageNode. Restores the
standalone-image output that the camera-projection path previously offered,
now unified through the render graph for both projection modes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The output can now be a mesh or a standalone texture image, so the
mesh-specific option name is misleading. Keeps the -o short flag.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
setProjectionParams() no longer forces ProjectionMode::Camera; it only
stores the explicit camera. Callers set the mode explicitly via
setProjectionMode(). Add clearProjectionParams() to drop an explicit camera
and revert to auto-derivation.

This removes the side effect that made ReorderTextureNode's cache
(de)serialization fragile: deserialize now restores explicit params before
the mode (so the serialized mode is authoritative) and clears stale params
when none were cached.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add rt::ValidateProjectionParams(), which rejects non-positive/non-finite
focal lengths and image sizes, non-finite principal points, and extrinsics
whose rotation block is not a proper rotation (orthonormal + det == +1).

create_texture_camera_() now validates the resolved camera (explicit or
auto-derived) and throws on failure, replacing the size-only guard and
covering the degenerate auto-camera case. ParseCameraFile() validates user
camera files up front for a clean CLI error. Adds a GoogleTest covering the
accept/reject cases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extract the per-hit UV->color sampling shared by the orthographic and camera
paths into sample_surface_color_(). Each loop now hoists a single
inputTexture_.empty() check and only samples color when a texture is present,
so camera mode (and the orthographic path) no longer call getRectSubPix on an
empty image; depth/position maps are still produced.

Also document that CreateProjectiveUVMap does no near-plane clipping, so
camera-straddling triangles map incorrectly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Warn when --camera-file is supplied without --projection camera (instead of
silently ignoring it), and note the dependency in the option help. Also note
in --depth-map/--position-map help that their float output wants a
float-capable format such as .tif.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@csparker247

Copy link
Copy Markdown
Member Author

Addressed the review findings in four focused commits:

  • bb628cb Decouple setProjectionParams from mode — it no longer force-sets Camera; added clearProjectionParams() to revert to auto-derivation. Node deserialize_ now restores params before mode (serialized mode is authoritative) and clears stale params when none were cached. (Fixes the serialize/deserialize coupling.)
  • c07cf2d Validate pinhole camera params — new rt::ValidateProjectionParams() rejects non-positive/non-finite fx/fy, non-positive image size, non-finite principal point, and extrinsics whose rotation block isn't a proper rotation (R·Rᵀ ≈ I and det ≈ +1). Wired into create_texture_camera_() (covers the degenerate auto-camera too) and ParseCameraFile() (clean CLI error). Added a GoogleTest (rt_TestReorderUnorganizedTexture, 7 cases). (Fixes fx/fy guard + non-orthonormal pose + zero-extent auto-camera.)
  • 33db713 Share color sampling — extracted the duplicated per-hit UV→color block into sample_surface_color_(); each loop hoists one inputTexture_.empty() check so neither path calls getRectSubPix on an empty image (depth/position still produced). Documented the no-near-plane-clipping limitation in CreateProjectiveUVMap. (Fixes dedup + empty-texture.)
  • 60d60b0 CLI polish — warn when --camera-file is set without --projection camera (+ note in help); note float-capable format (.tif) in --depth-map/--position-map help.

On the float-map finding: WriteImage already warns when writing float to a non-float format and routes .tif through a float-capable writer, so no code-path change was needed beyond the help note.

Full build is clean and all 4 tests pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant