Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

#### 👗 Virtual try-on — Google Vertex (`virtual-try-on-001`)
- **Google Virtual Try-On** (`GoogleVTONAdapter` / CLI `--model google-vton`): first-party Vertex / Gemini Enterprise `recontext_image`
- Dedicated person + product try-on; **not** Nano Banana and **not** `GEMINI_API_KEY`
- Auth: `GOOGLE_CLOUD_PROJECT` + Application Default Credentials (`gcloud auth application-default login` or `GOOGLE_APPLICATION_CREDENTIALS`)
- 1–4 samples; C2PA/SynthID watermark on by default; MCP tool `vton_google_vton`
- Docs: `docs/docs/api-reference/google-vton.md`

#### 🧠 Understand + 🎬 Video — NVIDIA NIM (Nemotron Omni, Cosmos 3)
- **Nemotron 3 Nano Omni** (`NemotronOmniUnderstandAdapter` / CLI `--model nemotron-omni`): hosted NIM chat (`nvidia/nemotron-3-nano-omni-30b-a3b-reasoning`)
- Image, video, and audio understand; `NVIDIA_API_KEY`; thinking on by default
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ API tutorials, configuration, examples, and agent guides live there — not in t

| Category | Highlights |
|---|---|
| **Virtual try-on** | FLUX VTO, Nova Canvas, Kling AI, Segmind, Pruna P-Image-Try-On, FASHN, Nano Banana 2 Lite, **Qwen-Image** (API + local), **Muse Image** (composition) |
| **Virtual try-on** | FLUX VTO, **Google Vertex VTO**, Nova Canvas, Kling AI, Segmind, Pruna P-Image-Try-On, FASHN, Nano Banana 2 Lite, **Qwen-Image** (API + local), **Muse Image** (composition) |
| **Generate / edit** | Nano Banana family, FLUX.2, GPT Image, Luma Photon, Seedream 5.0 Pro, Ideogram 4.0, Grok Imagine Image, Pruna P-Image / **P-Image-Ideogram** / Edit / Upscale, **Qwen-Image** (API + local), **Muse Image**; local FLUX.2-dev Turbo |
| **Understand** | Kimi K2.6 / K2.7 Code / K3 (API), Kimi-VL & LLaVA-NeXT (local), **Qwen3.8-Max** (API) + **Qwen3.8-27B** (local) |
| **Video** | Veo, Sora, Luma Ray 2 + Ray 3.2, Seedance 2.5, Kling 3.0 / Omni / Turbo, Grok Imagine Video 1.5, Gemini Omni Flash, Pruna P-Video / Replace / Avatar / Animate, **LTX-2.5** (API + local), **Hailuo 2.3**, **MiniMax H3** (API + local), **Wan** (API + local 2.2), **Runway Gen-4.5** |
Expand Down
6 changes: 3 additions & 3 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ This roadmap tracks what shipped and what comes next. Product strategy: [`VISION
│ • Fashion prompt packs + train-pack schema / datasets │
│ • Train / LoRA recipes (`opentryon train`) + notebooks │
│ • Eval + Fashion Bench v0 (`opentryon eval`) │
│ • One productized local OSS VTON path (CatVTON / IDM / OOT — pick one) │
│ • One productized local OSS VTON path (Leffa / CatVTON / IDM / OOT — pick one) │
│ • Fashion agentic workflows via MCP (Try-On QA or Fine-Tune Coach) │
│ • Efficiency card for that local path (VRAM / latency table) │
└─────────────────────────────────────────────────────────────────────────┘
Expand Down Expand Up @@ -98,7 +98,7 @@ CLI/MCP remain the distribution spine; new capabilities register the same way.
- [ ] CLI: `opentryon eval run|report`

### Slice D — Local OSS VTON (productize one)
- [ ] Ship **one** of CatVTON / IDM-VTON / OOTDiffusion under `tryon.models` + `opentryon[local]`
- [ ] Ship **one** of Leffa / CatVTON / IDM-VTON / OOTDiffusion under `tryon.models` + `opentryon[local]` (candidates + licenses: [integrate-next](docs/docs/community/integrate-next.md))
- [ ] Same invoke path as cloud adapters (agents don’t care where it runs)
- [ ] Docs: install, VRAM, dry-run, known limits

Expand Down Expand Up @@ -179,7 +179,7 @@ tryon/

**High-value next:** LoRA recipe hardening, Fashion Bench metrics, first local VTON adapter, Try-On QA or Fine-Tune Coach workflow.

**Vendor / model queue** (not a v0.1.0 commitment): [docs/docs/community/integrate-next.md](docs/docs/community/integrate-next.md) — NVIDIA Nemotron Omni + Cosmos 3 first, then CatVTON, then new services (LipSync, TRELLIS).
**Vendor / model queue** (not a v0.1.0 commitment): [docs/docs/community/integrate-next.md](docs/docs/community/integrate-next.md) — NVIDIA Wave 1 and Google Vertex VTO shipped; next is OutfitAnyone-Plus and one local VTON weight, then new services (LipSync, TRELLIS).

See [Contributing](CONTRIBUTING.md), [new-model checklist](docs/docs/advanced/new-model-checklist.md), and [VISION.md](VISION.md).

Expand Down
2 changes: 2 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,8 @@ KLING_AI_API_KEY
KLING_AI_SECRET_KEY
SEGMIND_API_KEY
GEMINI_API_KEY
GOOGLE_CLOUD_PROJECT
GOOGLE_APPLICATION_CREDENTIALS
BFL_API_KEY
LUMA_AI_API_KEY
OPENAI_API_KEY
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/agents/planner-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Each capability has a **default model**. The planner uses that default unless th
| Video generation | `video` | `video-generate` / `sora` |
| Background remove | `bg_remove` | `bg-remove` / `ben2` |

- **User named a model** (for example `wan-3.0`, `flux2-pro`, `p-image-ideogram`, `nemotron-omni`, `cosmos3`) → that registry id **only**, even if it lives in another capability. An unknown name is **not** replaced by the default; the planner asks you to pick a real id. `p-image-ideogram` pins Pruna’s model; a bare `ideogram` still pins Ideogram 4.0. `cosmos3-reasoner` pins the VLM, not the Generator.
- **User named a model** (for example `wan-3.0`, `google-vton`, `flux2-pro`, `p-image-ideogram`, `nemotron-omni`, `cosmos3`) → that registry id **only**, even if it lives in another capability. An unknown name is **not** replaced by the default; the planner asks you to pick a real id. `p-image-ideogram` pins Pruna’s model; a bare `ideogram` still pins Ideogram 4.0. `cosmos3-reasoner` pins the VLM, not the Generator. `google-vton` / `virtual-try-on-001` pin Vertex dedicated try-on, not Nano Banana.
- **No model named** → the default in the table.
- The classifier must leave `model` empty unless the user named one. A leaked default in `plan.model` is ignored unless that id also appears in the prompt.

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/agents/vton-agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ keywords:

`VTOnAgent` is a **thin Python facade** over the live registry. It is not a LangChain tool loop. For Studio chat, call [`PlannerAgent`](./planner-agent.md) / MCP `planner_agent`.

Default model: **`kling-ai`**. A named model in the prompt (FASHN, FLUX VTO, Segmind, Qwen-Image, …) pins that VTON registry id.
Default model: **`kling-ai`**. A named model in the prompt (Google VTO, FASHN, FLUX VTO, Segmind, Qwen-Image, …) pins that VTON registry id.

```python
from tryon.agents.vton import VTOnAgent
Expand Down
73 changes: 73 additions & 0 deletions docs/docs/api-reference/google-vton.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
---
sidebar_position: 8
title: Google Virtual Try-On
description: Vertex AI virtual-try-on-001 — dedicated person + product try-on (not Gemini API / Nano Banana)
---

# Google Virtual Try-On

First-party Vertex / Gemini Enterprise model **`virtual-try-on-001`**. Person photo + product photo → try-on stills (1–4 samples). Output aspect and resolution match the person image. PNG/JPEG, 10MB max. C2PA / SynthID watermark on by default.

This is **not** Nano Banana composition (`nano-banana-2-lite` / `GEMINI_API_KEY`). The Gemini Developer API does not host this model.

| CLI `--model` | MCP tool | Adapter | Upstream id |
|---|---|---|---|
| `google-vton` | `vton_google_vton` | `GoogleVTONAdapter` | `virtual-try-on-001` |

Official docs:

- [Virtual Try-On 001 (Vertex)](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/models/imagen/virtual-try-on-001)
- [Generate Virtual Try-On Images](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/capabilities/generate-virtual-try-on-images)

GA 20 January 2026. Google lists a discontinuation date of 20 January 2027 for this model id — check Vertex docs before relying on it past that.

## Auth

Vertex **Application Default Credentials** plus a GCP project. Studio Connect stores the project id only.

```bash
gcloud auth application-default login
export GOOGLE_CLOUD_PROJECT=your-gcp-project
# optional; default is global
# export GOOGLE_CLOUD_LOCATION=global
# or a service account:
# export GOOGLE_APPLICATION_CREDENTIALS=/path/to/sa.json
```

Enable billing and the Vertex / Agent Platform APIs on that project. If `global` 404s in your region, set `GOOGLE_CLOUD_LOCATION=us-central1`.

## CLI

No text prompt — the API rejects styling instructions. Shopper photos need `--person-generation allow_adult` (the default).

```bash
opentryon vton --model google-vton \
--person-image person.jpg \
--garment-image sweater.jpg

opentryon vton --model google-vton \
--person-image person.jpg \
--garment-image sweater.jpg \
--num-images 2 \
--seed 7 \
--dry-run
```

## Python

```python
from tryon.api import GoogleVTONAdapter

adapter = GoogleVTONAdapter() # GOOGLE_CLOUD_PROJECT
images = adapter.generate_and_decode(
person="person.jpg",
garment="sweater.jpg",
number_of_images=1,
person_generation="allow_adult",
)
images[0].save("worn.png")
```

## Planner / Studio

Name **`google-vton`** or **`virtual-try-on-001`** in chat to pin this model. The VTON default stays `kling-ai`. After MCP restart, Connect lists **Google Vertex Virtual Try-On** (`GOOGLE_CLOUD_PROJECT`). ADC still lives on the MCP host, not in Studio.
2 changes: 2 additions & 0 deletions docs/docs/api-reference/nano-banana.md
Original file line number Diff line number Diff line change
Expand Up @@ -376,6 +376,8 @@ images[0].save("result.png")

Also available from the `opentryon` CLI / MCP server as `vton --model nano-banana-2-lite`.

This is **composition** try-on via the Gemini Developer API (`GEMINI_API_KEY`). Dedicated Google Cloud Virtual Try-On is `--model google-vton` (Vertex `virtual-try-on-001`, ADC + `GOOGLE_CLOUD_PROJECT`). See [Google Virtual Try-On](google-vton).

## Command Line Usage

Use the `image_gen.py` script for command-line image generation:
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/api-reference/nvidia-nim.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ One key, `NVIDIA_API_KEY` from [build.nvidia.com](https://build.nvidia.com), unl
| `cosmos3-reasoner` | `understand` | `Cosmos3ReasonerAdapter` | `nvidia/cosmos3-nano-reasoner` |
| `cosmos3` | `video-generate` | `Cosmos3VideoAdapter` | Cosmos 3 Generator nano (`POST` infer, `b64_video`) |

Nemotron is **understanding / agents**, not T2I or VTON. Cosmos 3 Generator is **physics-aware T2V / I2V**. There is no NVIDIA VTON NIM — local OSS VTON stays on the [integrate-next](../community/integrate-next.md) Wave 2 list.
Nemotron is **understanding / agents**, not T2I or VTON. Cosmos 3 Generator is **physics-aware T2V / I2V**. There is no NVIDIA VTON NIM — dedicated try-on APIs and local weights stay on the [integrate-next](../community/integrate-next.md) Wave 2 list.

Official docs:

Expand Down
27 changes: 27 additions & 0 deletions docs/docs/api-reference/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,33 @@ See [Kling AI API Documentation](kling-ai) for complete details.

---

### `GoogleVTONAdapter`

Dedicated Vertex AI Virtual Try-On (`virtual-try-on-001`). Person + product images; **not** Gemini API / Nano Banana.

```python
from tryon.api import GoogleVTONAdapter

adapter = GoogleVTONAdapter() # GOOGLE_CLOUD_PROJECT + ADC

images = adapter.generate_and_decode(
person="person.jpg",
garment="sweater.jpg",
number_of_images=1,
)
```

**Parameters:**
- `project` (str, optional): GCP project. Defaults to `GOOGLE_CLOUD_PROJECT`
- `location` (str, optional): Vertex location. Defaults to `GOOGLE_CLOUD_LOCATION` or `global`

**Methods:**
- `generate_and_decode(person, garment, ...)` - Generate and decode to PIL Images

See [Google Virtual Try-On](google-vton) for auth, CLI, and MCP.

---

### `AmazonNovaCanvasVTONAdapter`

Adapter for Amazon Nova Canvas Virtual Try-On through AWS Bedrock.
Expand Down
Loading
Loading