Image-to-3D with a training-free, tree-aware HiCache velocity cache on the slat-stage flow matching.
A fork of SAM 3D Objects (Meta) that skips most network evaluations in the slat-stage flow-matching sampler by forecasting the cached velocity with a scaled-Hermite polynomial (HiCache) — generalised to SAM3D's PyTree (structured) velocities.
These repos are complementary accelerators, not competing solutions — each speeds up a different
base generator, and the + / ++ suffix is a method choice, not a rival product. Pick by
(1) which base model you run, then (2) which forecast basis you want:
| base generator | + = HiCache (Hermite) |
++ = HiCache++ (DMD) |
|---|---|---|
| Hunyuan3D-2.1 | hunyuan2.1-plus |
hunyuan2.1-plus-plus |
| Hunyuan3D-2 mini | hunyuan2-plus |
hunyuan2-plus-plus |
| SAM 3D Objects | sam3d-plus |
sam3d-plus-plus |
| Fast-SAM3D | fastsam3d-plus |
fastsam3d-plus-plus |
| TRELLIS (v1) | faster-trellis |
faster-trellis-plus-plus |
| TRELLIS.2-4B (v2) | hermit-trellis2 |
hermit-trellis2-plus-plus |
+(HiCache / scaled-Hermite): the published polynomial velocity-forecast basis — conservative, reproduces the HiCache paper. Use it to deploy the established method.++(HiCache++ / DMD exponential): our Dynamic-Mode-Decomposition basis — the same near-lossless quality at wider skip intervals, where the polynomial diverges. Use it when you push the cache interval for more speed.- standalone / model-agnostic:
hicache-plus-plus— the forecaster itself, to add DMD caching to your own diffusion/flow model. fast-trellis2= the TaylorSeer baseline fork (the upstream "Fast" accel) — the v2 reference point, not a HiCache variant.
This repo:
sam3d-plus— SAM 3D Objects × HiCache (Hermite) — slat-stage FlowMatching cache.
SAM 3D Objects reconstructs full 3D shape, texture, and layout from a single masked image. The expensive part of inference is the slat-stage flow-matching sampler: an Euler ODE solve that calls the backbone once per step over many steps.
This fork adds HiCache to that sampler — training-free, geometry-preserving feature caching. On
most solver steps it skips the backbone and instead forecasts the (CFG-combined) velocity from
cached anchors, calling the network only every interval steps. The single twist over upstream HiCache
is that SAM3D's solver state and backbone velocity are torch.utils._pytree structures, not single
tensors, so the cache is tree-aware: the Hermite/finite-difference coefficients are scalars shared by
all leaves, and a forecast is one tree_map per order.
This is the HiCache (Hermite polynomial) fork. For the exponential (DMD/Prony) forecaster that stays lossless out to a larger skip interval, see the sibling fork
sam3d-plus-plusand the standalone libraryhicache-plus-plus.
A flow-matching sampler integrates dx/dt = v_θ(x, t). HiCache treats the per-step velocity as a smooth
trajectory in the step index and, on skipped steps, extrapolates it from cached finite-difference
derivatives using a dual-scaled physicist's-Hermite polynomial instead of evaluating v_θ — skipping
(interval-1)/interval of the backbone passes. Here the velocity is a PyTree, so each derivative order is
stored as a tree and the scalar Hermite coefficient H̃_n(k)/n! is applied leafwise via tree_map
(accel.py). A complementary Adaptive-CFG path drops the unconditional pass once conditional and
unconditional velocities align and rebuilds the guidance term g = w·(y_cond − y_uncond) from cached
anchors. The hook is native — the Euler solver and CFG module call the cache helpers directly; there
is no runtime monkey-patching.
HiCache lives on the Euler solver of the slat-stage FlowMatching module and is exposed as a convenience
method on the flow-matching object (flow_matching/model.py, flow_matching/solver.py):
# fm is the slat-stage FlowMatching module inside the SAM 3D Objects pipeline.
# HiCache requires the Euler solver (one dynamics_fn eval per step).
fm.enable_hicache(
interval=3, # call the backbone every 3rd step; forecast the other 2
max_order=1, # Hermite/finite-difference order
first_enhance=2, # always run full for the first 2 (warm-up) steps
end_enhance=None, # always run full for the final step(s); None = last step
sigma=0.5, # Hermite scale (0,1)
)
# optional, composable: drop the unconditional CFG pass once it aligns
fm.enable_adaptive_guidance(gamma_bar=0.94, warmup=2)
# ... run the normal SAM 3D Objects inference (generate / demo.py) ...
fm.disable_hicache() # restore the exact uncached sampler
fm.disable_adaptive_guidance()Internally enable_hicache is also available directly on the solver (ODESolver.enable_hicache(...)); the
solver resets the per-trajectory cache at the start of every run and only activates it for Euler. A CPU
unit test that needs no GPU or model weights ships in the accel module:
python -m sam3d_objects.model.backbone.generator.flow_matching.accelOn the slat-stage FlowMatching (real SAM 3D Objects weights, F1 vs the uncached baseline), HiCache
(Hermite) is geometry-lossless (F1 = 1.000) out to interval-3 at ~1.44×. The polynomial basis drifts
beyond that, which is exactly the skip ceiling the exponential basis is built to push.
| config | speedup | F1 vs baseline |
|---|---|---|
| vanilla (uncached) | 1.00× | 1.000 |
| HiCache i3 | 1.44× | 1.000 |
The vendored Hermite forecast in accel.py evaluated the basis at x = -k; the corrected
convention from hicache-plus-plus 1.2.0
is x = +k (the upstream TaylorSeer distance convention; -k flips every odd-order term).
This fork now ships the corrected forecast. The published table above was measured with the
as-released code and remains valid as-measured.
Re-validated on the same protocol as the published table (the runnable slat-stage
FlowMatching via Fast-SAM3D's InferencePipelinePointMap, identical vendored port, real
weights, seed 42, F1@0.05 of the output gaussians vs the uncached baseline), including
wider-interval probes where a polynomial-basis error would be expected to bite:
| config | as-released (x = -k) |
corrected (x = +k) |
|---|---|---|
| HiCache i3/o2 | F1 = 1.000 (CD 0.0125) | F1 = 1.000 (CD 0.0121) |
| HiCache i5/o3 | F1 = 1.000 (CD 0.0127) | F1 = 1.000 (CD 0.0126) |
| HiCache i6/o3 | F1 = 1.000 (CD 0.0128) | F1 = 1.000 (CD 0.0128) |
The corrected forecast matches the as-released result at the published interval (verdict: same, not worse), and on this stage the geometry is lossless at the F1@0.05 metric out to interval-6 under both conventions (speedups 1.3x to 1.5x; run-to-run latency noise dominates the timing differences). The Chamfer drift is marginally lower with the corrected sign at i3 and i5.
For the exponential (DMD/Prony) forecaster that holds F1 = 1.000 out to interval-6 at 1.56× on the
same stage — where Hermite is lossless only to interval-3 — see the sibling fork sam3d-plus-plus and
the standalone library hicache-plus-plus (with the controlled forecast
microbenchmark and the full Hunyuan3D / SAM3D tables).
- SAM 3D Objects © Meta Platforms, Inc. — model, weights, and code under the SAM License (note its restrictions: research/responsible-use terms, a publication-acknowledgement requirement, and trade-control / ITAR / sanctions compliance — not a permissive or unconditionally commercial license). The full upstream README (install, demos, benchmark, citation) is preserved below.
- HiCache — scaled-Hermite velocity forecasting, arXiv:2508.16984. The Hermite basis here is a clean reimplementation, generalised to PyTree velocities.
- Adaptive Guidance — arXiv:2312.12487.
The acceleration code added by this fork lives in
sam3d_objects/model/backbone/generator/flow_matching/{accel,solver,model}.py.
If you use this fork, please cite the base model and the acceleration methods it builds on.
SAM 3D Objects (base model):
@article{sam3dteam2025sam3d3dfyimages,
title={SAM 3D: 3Dfy Anything in Images},
author={SAM 3D Team and Xingyu Chen and Fu-Jen Chu and Pierre Gleize and Kevin J Liang and Alexander Sax and Hao Tang and Weiyao Wang and Michelle Guo and Thibaut Hardin and Xiang Li and Aohan Lin and Jiawei Liu and Ziqi Ma and Anushka Sagar and Bowen Song and Xiaodong Wang and Jianing Yang and Bowen Zhang and Piotr Dollár and Georgia Gkioxari and Matt Feiszli and Jitendra Malik},
year={2025},
eprint={2511.16624},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2511.16624},
}HiCache (scaled-Hermite velocity forecasting):
@misc{hicache2025,
title={HiCache: Training-free Acceleration of Diffusion Models via Hermite Polynomial Feature Forecasting},
eprint={2508.16984},
archivePrefix={arXiv},
year={2025}
}Adaptive Guidance:
@misc{adaptiveguidance2023,
title={Adaptive Guidance: Training-free Acceleration of Conditional Diffusion Models},
eprint={2312.12487},
archivePrefix={arXiv},
year={2023}
}SAM 3D Objects is one part of SAM 3D, a pair of models for object and human mesh reconstruction. If you’re looking for SAM 3D Body, click here.
SAM 3D Team, Xingyu Chen*, Fu-Jen Chu*, Pierre Gleize*, Kevin J Liang*, Alexander Sax*, Hao Tang*, Weiyao Wang*, Michelle Guo, Thibaut Hardin, Xiang Li⚬, Aohan Lin, Jia-Wei Liu, Ziqi Ma⚬, Anushka Sagar, Bowen Song⚬, Xiaodong Wang, Jianing Yang⚬, Bowen Zhang⚬, Piotr Dollár†, Georgia Gkioxari†, Matt Feiszli†§, Jitendra Malik†§
Meta Superintelligence Labs
*Core contributor (Alphabetical, Equal Contribution), ⚬Intern, †Project leads, §Equal Contribution
[Paper] [Code] [Website] [Demo] [Blog] [BibTeX] [Roboflow]
SAM 3D Objects is a foundation model that reconstructs full 3D shape geometry, texture, and layout from a single image, excelling in real-world scenarios with occlusion and clutter by using progressive training and a data engine with human feedback. It outperforms prior 3D generation models in human preference tests on real-world objects and scenes. We released code, weights, online demo, and a new challenging benchmark.
- 06/02/2026 - 3D Artist Object Set and HF Leaderboard are out.
- 06/01/2026 - Encoder weights are out.
- 11/19/2025 - Checkpoints Launched, Web Demo and Paper are out.
Follow the setup steps before running the following.
SAM 3D Objects can convert masked objects in an image, into 3D models with pose, shape, texture, and layout. SAM 3D is designed to be robust in challenging natural images, handling small objects and occlusions, unusual poses, and difficult situations encountered in uncurated natural scenes like this kidsroom:
For a quick start, run python demo.py or use the the following lines of code:
import sys
# import inference code
sys.path.append("notebook")
from inference import Inference, load_image, load_single_mask
# load model
tag = "hf"
config_path = f"checkpoints/{tag}/pipeline.yaml"
inference = Inference(config_path, compile=False)
# load image and mask
image = load_image("notebook/images/shutterstock_stylish_kidsroom_1640806567/image.png")
mask = load_single_mask("notebook/images/shutterstock_stylish_kidsroom_1640806567", index=14)
# run model
output = inference(image, mask, seed=42)
# export gaussian splat
output["gs"].save_ply(f"splat.ply")For more details and multi-object reconstruction, please take a look at out two jupyter notebooks:
SAM 3D Body (3DB) is a robust promptable foundation model for single-image 3D human mesh recovery (HMR).
As a way to combine the strengths of both SAM 3D Objects and SAM 3D Body, we provide an example notebook that demonstrates how to combine the results of both models such that they are aligned in the same frame of reference. Check it out here.
The SAM 3D Objects model checkpoints and code are licensed under SAM License.
See contributing and the code of conduct.
The SAM 3D Objects project was made possible with the help of many contributors.
Robbie Adkins, Paris Baptiste, Karen Bergan, Kai Brown, Michelle Chan, Ida Cheng, Khadijat Durojaiye, Patrick Edwards, Daniella Factor, Facundo Figueroa, Rene de la Fuente, Eva Galper, Cem Gokmen, Alex He, Enmanuel Hernandez, Dex Honsa, Leonna Jones, Arpit Kalla, Kris Kitani, Helen Klein, Kei Koyama, Robert Kuo, Vivian Lee, Alex Lende, Jonny Li, Kehan Lyu, Faye Ma, Mallika Malhotra, Sasha Mitts, William Ngan, George Orlin, Peter Park, Don Pinkus, Roman Radle, Nikhila Ravi, Azita Shokrpour, Jasmine Shone, Zayida Suber, Phillip Thomas, Tatum Turner, Joseph Walker, Meng Wang, Claudette Ward, Andrew Westbury, Lea Wilken, Nan Yang, Yael Yungster
If you use SAM 3D Objects in your research, please use the following BibTeX entry.
@article{sam3dteam2025sam3d3dfyimages,
title={SAM 3D: 3Dfy Anything in Images},
author={SAM 3D Team and Xingyu Chen and Fu-Jen Chu and Pierre Gleize and Kevin J Liang and Alexander Sax and Hao Tang and Weiyao Wang and Michelle Guo and Thibaut Hardin and Xiang Li and Aohan Lin and Jiawei Liu and Ziqi Ma and Anushka Sagar and Bowen Song and Xiaodong Wang and Jianing Yang and Bowen Zhang and Piotr Dollár and Georgia Gkioxari and Matt Feiszli and Jitendra Malik},
year={2025},
eprint={2511.16624},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2511.16624},
}
Model weights and demo/example assets are not committed to this repo — only the acceleration architecture (code + integration). Download the base-model weights from the upstream project, facebookresearch/sam-3d-objects, per its instructions, and point the loader at them (see the code / upstream README). This keeps the repository lightweight and avoids redistributing third-party weights.
Part of the HiCache++ acceleration family.
- Family hub:
hicache-plus-plus— the basis library behind this adapter. - Sibling:
sam3d-plus-plus— the same base model with the HiCache++ (Dynamic Mode Decomposition / Prony) exponential-forecast variant.