Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,10 +103,6 @@ uv run hf download appautomaton/sam-3d-objects-mlx --local-dir weights/sam-3d-ob
uv run mlx-spatial-sam3d validate weights/sam-3d-objects-mlx

uv run hf download appautomaton/lito-research-mlx --local-dir weights/lito-research-mlx
uv run hf download microsoft/TRELLIS-image-large \
ckpts/ss_dec_conv3d_16l8_fp16.json \
ckpts/ss_dec_conv3d_16l8_fp16.safetensors \
--local-dir weights/trellis2/microsoft/TRELLIS-image-large
uv run mlx-spatial-lito validate weights/lito-research-mlx
```

Expand Down
25 changes: 13 additions & 12 deletions docs/lito.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,6 @@ Recommended runtime bundle:
```bash
uv run hf download appautomaton/lito-research-mlx \
--local-dir weights/lito-research-mlx
uv run hf download microsoft/TRELLIS-image-large \
ckpts/ss_dec_conv3d_16l8_fp16.json \
ckpts/ss_dec_conv3d_16l8_fp16.safetensors \
--local-dir weights/trellis2/microsoft/TRELLIS-image-large
uv run mlx-spatial-lito validate weights/lito-research-mlx
uv run mlx-spatial-lito inspect weights/lito-research-mlx --limit 10
```
Expand All @@ -34,16 +30,19 @@ Expected converted layout:
```text
weights/lito-research-mlx/tokenizer/lito_new.safetensors
weights/lito-research-mlx/image_to_3d/lito_dit_rgba.safetensors
weights/trellis2/microsoft/TRELLIS-image-large/ckpts/ss_dec_conv3d_16l8_fp16.json
weights/trellis2/microsoft/TRELLIS-image-large/ckpts/ss_dec_conv3d_16l8_fp16.safetensors
weights/lito-research-mlx/dependencies/trellis/LICENSE
weights/lito-research-mlx/dependencies/trellis/SOURCE.json
weights/lito-research-mlx/dependencies/trellis/ckpts/ss_dec_conv3d_16l8_fp16.json
weights/lito-research-mlx/dependencies/trellis/ckpts/ss_dec_conv3d_16l8_fp16.safetensors
```

The first two files are the LiTo bundle. The final two are the sparse-structure
decoder used to convert LiTo voxel latents into Gaussian initialization
coordinates. They come from `microsoft/TRELLIS-image-large`, not from the
`microsoft/TRELLIS.2-4B` bundle used by the separate TRELLIS.2 pipeline.
`mlx-spatial-lito validate` checks all four runtime files; `inspect` reads only
the two LiTo safetensors.
The embedded sparse-structure decoder converts LiTo voxel latents into Gaussian
initialization coordinates. It is the exact checkpoint from
`microsoft/TRELLIS-image-large` revision
`25e0d31ffbebe4b5a97464dd851910efc3002d96`, not a dependency on the separate
TRELLIS.2 pipeline. `mlx-spatial-lito validate` requires the bundle-local
decoder and never searches an external TRELLIS root. `inspect` reads only the
two LiTo safetensors.

Maintainers can print Apple CDN download commands and convert local `.ckpt` files:

Expand Down Expand Up @@ -85,6 +84,8 @@ checkpoint records the exact policy, logical tensor shapes, bit width, group
size, and affine mode in safetensors metadata. The normal LiTo loader detects
that metadata and executes packed matrices directly with MLX quantized matrix
multiplication; no Torch or intermediate dequantized checkpoint is involved.
The root-level quantizer copies the embedded TRELLIS decoder and its provenance
files into the output bundle unchanged.

Pass the new root to inference exactly as you would the full-precision root:

Expand Down
5 changes: 5 additions & 0 deletions docs/model-publishing.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,14 @@ The tracked model-card source lives under:

```text
model-cards/lito-research-mlx/
model-cards/lito-research-mlx-8bit/
```

LiTo is research-only and non-commercial under Apple's model license. The model repository must include `LICENSE_MODEL`, identify the safetensors files as an unofficial converted derivative, and avoid language that implies Apple endorsement.
Both LiTo variants must embed the required Microsoft TRELLIS sparse-structure
decoder under `dependencies/trellis/`, together with its MIT license and an
immutable source manifest. Published LiTo bundles must not depend on a sibling
TRELLIS checkout.

## What The Model Card Should Contain

Expand Down
88 changes: 88 additions & 0 deletions model-cards/lito-research-mlx-8bit/LICENSE_MODEL
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
Disclaimer: IMPORTANT: This Apple Machine Learning Research Model is
specifically developed and released by Apple Inc. ("Apple") for the sole purpose
of scientific research of artificial intelligence and machine-learning
technology. “Apple Machine Learning Research Model” means the model, including
but not limited to algorithms, formulas, trained model weights, parameters,
configurations, checkpoints, and any related materials (including
documentation).

This Apple Machine Learning Research Model is provided to You by
Apple in consideration of your agreement to the following terms, and your use,
modification, creation of Model Derivatives, and or redistribution of the Apple
Machine Learning Research Model constitutes acceptance of this Agreement. If You
do not agree with these terms, please do not use, modify, create Model
Derivatives of, or distribute this Apple Machine Learning Research Model or
Model Derivatives.

* License Scope: In consideration of your agreement to abide by the following
terms, and subject to these terms, Apple hereby grants you a personal,
non-exclusive, worldwide, non-transferable, royalty-free, revocable, and
limited license, to use, copy, modify, distribute, and create Model
Derivatives (defined below) of the Apple Machine Learning Research Model
exclusively for Research Purposes. You agree that any Model Derivatives You
may create or that may be created for You will be limited to Research Purposes
as well. “Research Purposes” means non-commercial scientific research and
academic development activities, such as experimentation, analysis, testing
conducted by You with the sole intent to advance scientific knowledge and
research. “Research Purposes” does not include any commercial exploitation,
product development or use in any commercial product or service.

* Distribution of Apple Machine Learning Research Model and Model Derivatives:
If you choose to redistribute Apple Machine Learning Research Model or its
Model Derivatives, you must provide a copy of this Agreement to such third
party, and ensure that the following attribution notice be provided: “Apple
Machine Learning Research Model is licensed under the Apple Machine Learning
Research Model License Agreement.” Additionally, all Model Derivatives must
clearly be identified as such, including disclosure of modifications and
changes made to the Apple Machine Learning Research Model. The name,
trademarks, service marks or logos of Apple may not be used to endorse or
promote Model Derivatives or the relationship between You and Apple. “Model
Derivatives” means any models or any other artifacts created by modifications,
improvements, adaptations, alterations to the architecture, algorithm or
training processes of the Apple Machine Learning Research Model, or by any
retraining, fine-tuning of the Apple Machine Learning Research Model.

* No Other License: Except as expressly stated in this notice, no other rights
or licenses, express or implied, are granted by Apple herein, including but
not limited to any patent, trademark, and similar intellectual property rights
worldwide that may be infringed by the Apple Machine Learning Research Model,
the Model Derivatives or by other works in which the Apple Machine Learning
Research Model may be incorporated.

* Compliance with Laws: Your use of Apple Machine Learning Research Model must
be in compliance with all applicable laws and regulations.

* Term and Termination: The term of this Agreement will begin upon your
acceptance of this Agreement or use of the Apple Machine Learning Research
Model and will continue until terminated in accordance with the following
terms. Apple may terminate this Agreement at any time if You are in breach of
any term or condition of this Agreement. Upon termination of this Agreement,
You must cease to use all Apple Machine Learning Research Models and Model
Derivatives and permanently delete any copy thereof. Sections 3, 6 and 7 will
survive termination.

* Disclaimer and Limitation of Liability: This Apple Machine Learning Research
Model and any outputs generated by the Apple Machine Learning Research Model
are provided on an “AS IS” basis. APPLE MAKES NO WARRANTIES, EXPRESS OR
IMPLIED, INCLUDING WITHOUT LIMITATION THE IMPLIED WARRANTIES OF
NON-INFRINGEMENT, MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE,
REGARDING THE APPLE MACHINE LEARNING RESEARCH MODEL OR OUTPUTS GENERATED BY
THE APPLE MACHINE LEARNING RESEARCH MODEL. You are solely responsible for
determining the appropriateness of using or redistributing the Apple Machine
Learning Research Model and any outputs of the Apple Machine Learning Research
Model and assume any risks associated with Your use of the Apple Machine
Learning Research Model and any output and results. IN NO EVENT SHALL APPLE BE
LIABLE FOR ANY SPECIAL, INDIRECT, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
IN ANY WAY OUT OF THE USE, REPRODUCTION, MODIFICATION AND/OR DISTRIBUTION OF
THE APPLE MACHINE LEARNING RESEARCH MODEL AND ANY OUTPUTS OF THE APPLE MACHINE
LEARNING RESEARCH MODEL, HOWEVER CAUSED AND WHETHER UNDER THEORY OF CONTRACT,
TORT (INCLUDING NEGLIGENCE), STRICT LIABILITY OR OTHERWISE, EVEN IF APPLE HAS
BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

* Governing Law: This Agreement will be governed by and construed under the laws
of the State of California without regard to its choice of law principles. The
Convention on Contracts for the International Sale of Goods shall not apply to
the Agreement except that the arbitration clause and any arbitration hereunder
shall be governed by the Federal Arbitration Act, Chapters 1 and 2.

Copyright (C) 2026 Apple Inc. All Rights Reserved.
121 changes: 121 additions & 0 deletions model-cards/lito-research-mlx-8bit/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
---
license: other
license_name: apple-machine-learning-research-model-license-agreement
license_link: https://github.com/apple/ml-lito/blob/main/LICENSE_MODEL
library_name: mlx
pipeline_tag: image-to-3d
base_model:
- appautomaton/lito-research-mlx
tags:
- mlx
- apple-silicon
- safetensors
- image-to-3d
- gaussian-splatting
- 3dgs
- 8-bit
- affine-quantization
- runtime-only
- research-only
- non-commercial
---

# LiTo Runtime 8-bit Affine for `mlx-spatial`

A self-contained LiTo inference bundle with selective affine INT8 weights for
Apple Silicon. Accuracy-sensitive boundaries remain in FP32. This is an
unofficial derivative for non-commercial research use, not an Apple release.

## Use

```bash
pip install \
"mlx-spatial @ git+https://github.com/appautomaton/mlx-spatial.git@afa6b6512567dc9964294a1cec8601e1b505802e"

hf download appautomaton/lito-research-mlx-8bit \
--local-dir weights/lito-research-mlx-8bit

mlx-spatial-lito validate weights/lito-research-mlx-8bit

mlx-spatial-lito generate inputs/lito/object-rgba.png \
--weights-root weights/lito-research-mlx-8bit \
--output outputs/lito/object-8bit.ply \
--format ply \
--num-steps 20 \
--cfg-scale 3.0 \
--print-metrics
```

The output is a 3D Gaussian Splat PLY, not a triangle mesh. A clean RGBA
foreground matte is strongly recommended.

## Bundle

| File | Logical tensors | Quantized matrices | Bytes |
| --- | ---: | ---: | ---: |
| `image_to_3d/lito_dit_rgba.safetensors` | 1,016 | 224 | 2,004,347,727 |
| `tokenizer/lito_new.safetensors` | 467 | 124 | 168,537,103 |
| `dependencies/trellis/ckpts/ss_dec_conv3d_16l8_fp16.safetensors` | 74 | 0 | 147,591,972 |
| **Runtime weights** | **1,557** | **348** | **2,320,476,802** |

The bundle also includes the decoder config, Microsoft MIT license, and an
immutable source manifest under `dependencies/trellis/`.

The LiTo quantization scheme is affine 8-bit with group size 64. Packed weights
are stored as `uint32` with FP32 scales and biases and execute directly through
`mx.quantized_matmul`. Internal attention and MLP matrices in the EMA DiT,
Gaussian decoder, and voxel decoder are quantized. The image conditioner,
convolutions, embeddings, normalizations, boundary projections, and output
heads remain FP32.

## Embedded TRELLIS Decoder

LiTo uses the TRELLIS sparse-structure decoder to convert voxel latents into
Gaussian initialization coordinates. This bundle embeds the exact checkpoint
from `microsoft/TRELLIS-image-large` revision
`25e0d31ffbebe4b5a97464dd851910efc3002d96`:

```text
dependencies/trellis/LICENSE
dependencies/trellis/SOURCE.json
dependencies/trellis/ckpts/ss_dec_conv3d_16l8_fp16.json
dependencies/trellis/ckpts/ss_dec_conv3d_16l8_fp16.safetensors
```

The decoder safetensors SHA-256 is
`1c76d4a40519aa2d711cc263a8404105231ac26db31d946bed48b84fee79009a`.
The runtime does not search a separate TRELLIS checkout.

## Verification

- Both checkpoints pass `mlx-spatial-lito validate`.
- Architecture inspection recovers 28 DiT blocks, 6 Gaussian Perceiver blocks,
and 4 voxel decoder blocks.
- Real-weight Linear probes measured `0.51%–0.61%` relative RMSE and cosine
similarity above `0.99998` against FP32.
- An uncapped 20-step run produced 557,568 finite Gaussians in 2 minutes
39.76 seconds, with 11.60 GiB peak active MLX memory.
- A separate bundle-local 20-step validation produced 8,192 finite Gaussians
after explicitly capping occupied cells for packaging verification.

These figures are one local Apple Silicon observation, not a general benchmark
or an official quality-equivalence claim.

## Limitations and Licenses

- Inference only; training and mesh-specific modules are intentionally absent.
- Quantization can change generation details relative to FP32.
- Single-view reconstruction cannot determine unseen geometry with certainty.
- Commercial use is not permitted by Apple's model license.

LiTo weights are covered by the bundled `LICENSE_MODEL`. The embedded TRELLIS
decoder is covered by the MIT License under `dependencies/trellis/LICENSE`.

> Apple Machine Learning Research Model is licensed under the Apple Machine Learning Research Model License Agreement.

## Links

- [FP32 runtime variant](https://huggingface.co/appautomaton/lito-research-mlx)
- [`appautomaton/mlx-spatial`](https://github.com/appautomaton/mlx-spatial)
- [Apple LiTo project](https://apple.github.io/ml-lito/)
- [Apple LiTo source](https://github.com/apple/ml-lito)
21 changes: 21 additions & 0 deletions model-cards/lito-research-mlx-8bit/dependencies/trellis/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) Microsoft Corporation.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"files": {
"ckpts/ss_dec_conv3d_16l8_fp16.json": {
"bytes": 245,
"sha256": "646781293f1cda74720de85d1cef50a957fb4aebd9a4bd014e454e32f2330ac5"
},
"ckpts/ss_dec_conv3d_16l8_fp16.safetensors": {
"bytes": 147591972,
"sha256": "1c76d4a40519aa2d711cc263a8404105231ac26db31d946bed48b84fee79009a"
}
},
"license": "MIT",
"repo_id": "microsoft/TRELLIS-image-large",
"revision": "25e0d31ffbebe4b5a97464dd851910efc3002d96"
}
Loading