feat(blaze): implement template catalog - #2219
Conversation
github-actions
Bot
added
component:blaze
|
@codex Review 实现可持久化的 runtime artifact catalog，支持原子导入、列表和详情查询。 |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: df249214d3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
df24921 to
d3fdeec
Compare
|
@codex Review 实现可持久化的 runtime artifact catalog，支持原子导入、列表和详情查询。 |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d3fdeec9f5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
d3fdeec to
fa05a1f
Compare
|
@codex Review 实现可持久化的 runtime artifact catalog，支持原子导入、列表和详情查询。 |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fa05a1fd4e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex Review 实现可持久化的 runtime artifact catalog，支持原子导入、列表和详情查询。 |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fa05a1fd4e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex Review 实现可持久化的 runtime artifact catalog，支持原子导入、列表和详情查询。 |
|
Codex Review: Didn't find any major issues. Another round soon, please! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
fa05a1f to
5365034
Compare
WeissonHan
changed the title
|
@codex review 提供持久化 runtime template 的受限导入、原子发布、列表和查询能力，并保证目录操作始终绑定启动时验证的目录对象 |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5365034632
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
5365034 to
5b58d3e
Compare
|
@codex review 提供持久化 runtime template 的受限导入、原子发布、列表和查询能力，并保证目录操作始终绑定启动时验证的目录对象 |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5b58d3e46a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
![qoderai[bot]](https://avatars.githubusercontent.com/in/1396158?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5b58d3e46a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
5b58d3e to
f464519
Compare
|
@codex review 提供持久化 runtime template 的受限导入、原子发布、列表和查询能力，并保证目录操作始终绑定启动时验证的目录对象 |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f464519c66
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
86a03dd to
881f109
Compare
|
@codex Review 实现可持久化的 runtime artifact catalog，支持原子导入、列表和详情查询 |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 881f109ca8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review Please review the complete pull request: every commit, the cumulative diff against the base branch, and the implementation and documentation as one submitted change. Do not restrict the review to the most recently modified commit. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 881f109ca8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
881f109 to
048d4b5
Compare
|
@codex review Please review the complete pull request: every commit, the cumulative diff against the base branch, and the implementation and documentation as one submitted change. Do not restrict the review to the most recently modified commit. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 048d4b5eb6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
0881a16 to
01c8b8a
Compare
|
@codex review Please review the complete pull request: every commit, the cumulative diff against the base branch, and the implementation and documentation as one submitted change. Do not restrict the review to the most recently modified commit. |
|
Codex Review: Didn't find any major issues. Breezy! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
01c8b8a to
d67385e
Compare
|
@codex review Please review the complete pull request: every commit, the cumulative diff against the base branch, and the implementation and documentation as one submitted change. Do not restrict the review to the most recently modified commit. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d67385ed67
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
d67385e to
d64078f
Compare
|
@codex review Please review the complete pull request: every commit, the cumulative diff against the base branch, and the implementation and documentation as one submitted change. Do not restrict the review to the most recently modified commit. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d64078f4b3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
d64078f to
54cf646
Compare
|
@codex review Please review the complete pull request: every commit, the cumulative diff against the base branch, and the implementation and documentation as one submitted change. Do not restrict the review to the most recently modified commit. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 54cf6465c6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
54cf646 to
89bf81b
Compare
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 89bf81bd6a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Publish prepared runtime artifact sets and expose their metadata through /v1/templates. The durable catalog replaces the unused process-local registry and its GC route. Future sandbox-create work can resolve optional names from this same catalog. Imports validate bounded regular files and reserve catalog byte and entry capacity. They copy into private staging directories and publish without replacing an existing name. Each published entry remains independent after its prepared source changes or is removed. List reads validate entries one at a time and return sorted, name-only summaries. A response-owned permit prevents concurrent lists from retaining multiple response bodies. Single-item reads use an independent permit across metadata parsing and response delivery. Startup rejects catalog or import layouts that overlap daemon-owned peer resources. Configured paths, resolved targets, and Linux mount identities receive the same checks. Catalog artifacts are mount-checked on pinned handles before classification or readable open. Catalog entry directories are mount-checked before readable open and rechecked before use. Opened catalog, import, configuration, and artifact objects stay bound after validation. Absent catalog roots are created from a retained parent and reject components that appear during validation. Policy entry discovery keeps fail and warn startup modes aligned with policy loading. PATH-resolved host helpers are also excluded from catalog ownership. Failed staging setup is compensated; uncertain cleanup blocks later imports for recovery. Graceful shutdown cancels imports and waits for their resource claims to close. Sandbox creation, reference counting, and catalog deletion remain out of scope. Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com> Signed-off-by: Weisson Han <wenshu.hx@linux.alibaba.com>
Explain that /v1/templates is the daemon's single operator-facing template resource. Describe catalog configuration, artifact import, name discovery, and complete metadata lookup. The bilingual guides cover source requirements, capacity limits, private publication, and recovery. They record ownership boundaries, bounded reads, and independent list and item response budgets. Sandbox creation does not yet accept a template name. Future create support will resolve optional names from this catalog. It will not add another registry or API namespace. Reference counting, deletion, request bounds, and connection draining remain separate work. Signed-off-by: Jing Zhang <renyu.zj@linux.alibaba.com> Signed-off-by: Weisson Han <wenshu.hx@linux.alibaba.com>
89bf81b to
a283ec3
Compare
|
@codex review |
|
Codex Review: Didn't find any major issues. Keep it up! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Why
Blaze main exposed a process-local
/v1/templatesregistry, but the daemon had no production path that could publish an entry into it and its state disappeared on restart. Operators therefore could not import, discover, or inspect a reusable template.Adding a second durable template namespace would leave the public model ambiguous: operators would see two resources named “template”, while future sandbox creation would have no clear source of truth. This PR establishes one contract before clients depend on either interpretation:
/v1/templatesis the daemon's single operator-facing template catalog, and future sandbox-create support will resolve optional template names from this catalog.What changed
Before:
GET /v1/templates,GET /v1/templates/{id}, andPOST /v1/templates/gcrepresented an in-memory registry with no production publisher or durable entries.After: operators can import an independently copied template through
POST /v1/templates/import, list published names throughGET /v1/templates, and inspect one entry's complete metadata throughGET /v1/templates/{name}. The unused process-local registry and its GC route are removed, and one[template]configuration section owns the catalog directory, optional import root, and capacity limits.Imports accept only the documented bounded regular-file layout below the configured source root. The catalog reserves byte and entry capacity, copies into a private staging directory, synchronizes completed data, and publishes without replacing an existing name. Failed staging cleanup is synchronized; if cleanup cannot be confirmed, later imports are blocked until repair and restart.
List reads validate one entry at a time, retain only its bounded name summary, and discard complete metadata before validating the next entry. One permit covers the blocking scan and returned list body. A separate permit covers single-entry metadata parsing, serialization, and the returned item body. A second request of the same kind receives
503 Service Unavailableuntil the retained body is released; list and item budgets remain independent.Startup validates configured and resolved catalog boundaries before changing catalog permissions or contents. Opened catalog, import, configuration, entry-directory, and artifact objects remain attached to the identities accepted at startup. On Linux, catalog child directories and artifacts are pinned and mount-checked before a read-capable descriptor is derived, then rechecked against the same object before use. Generic directory setup does not reopen the retained catalog through its configured path. A warn-mode policy-entry discovery failure remains a sticky startup decision: the initial policy engine stays empty instead of rescanning a recovered directory. Graceful shutdown rejects new imports, cancels active copies, and waits for their resource claims to close.
391de162fimplements the single durable catalog, canonical/v1/templatesroutes, unified[template]configuration, bounded import/publication/recovery and lookup responses, startup ownership boundaries, validated catalog child handling, and regression tests that reject the retired duplicate and GC routes.a283ec345documents the single-resource model, operator workflow, capacity and boundary rules, bounded list and item reads, and the future sandbox-create contract in English and Chinese.These commits belong in one PR because the implementation and operator documentation define one independently usable API contract: import, publish, list, and inspect templates under a single resource. Sandbox creation from a catalog entry is deliberately not claimed here.
Related issue
closes #2216
Tracked under #1829.
User / Agent impact
Operators have one template resource to configure and monitor. Imported entries survive daemon restart and remain independent of later source-directory changes. Sandbox creation does not yet accept a template name; the planned create path will consume names from this same catalog rather than introduce another registry or namespace.
Risk and compatibility
This intentionally replaces the unused process-local API contract: item lookup changes from a UUID-shaped registry ID to a catalog name, and
POST /v1/templates/gcis removed. The old registry had no production publisher and held no durable entries, so there is no persisted registry data to convert. Existingtemplate.gc_intervalandtemplate.idle_ttlsettings no longer control behavior and should be removed;template.dirnow owns the published catalog.Related lifetime and load-binding limitations remain tracked separately:
Until those fixes land, deployments must keep the configured peer-root ancestors stable for the daemon lifetime and avoid changing policy sources between boundary validation and load or reload. Sandbox creation from catalog entries, reference counting, catalog deletion, successful-discovery policy-source binding and later policy reload validation, daemon-wide request-body limits, and daemon-wide connection draining remain outside this PR.
Validation
The source tree at the submitted head
a283ec345is identical (549aea266d67) to the candidate tree checked on Linux x86_64 using task-local source and build directories.All of the following passed:
cargo fmt --all --checkcargo clippy --all-targets --locked -- -D warningscargo clippy --all-targets --all-features --locked -- -D warningscargo test --locked: 52 blaze-core + 203 blazed, 0 failedcargo test --all-features --locked: 52 blaze-core + 213 blazed, 0 failedRUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps --lockedRUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps --all-features --lockedbash scripts/docs-lint.shpython3 scripts/docs-link-check.pyThe regression suite includes catalog-owner locking, creation through a retained parent when the catalog is absent, and protection of every executable helper candidate found through
PATH. Linux call-order tests prove that foreign-mount artifacts and catalog child directories are rejected from pinned descriptors before classification or read-capable open, and that the opened directory is rechecked before identity inspection. A name-replacement test proves directory reads remain attached to the pinned object; startup accounting reuses that same validated descriptor instead of reopening the name. Item-response tests verify that the permit is global across template names, remains owned by every response-body clone, survives cancellation until detached blocking work exits, releases after read errors, and is independent of the list-response permit; API coverage verifies the sameapplication/jsonresponse and503overload behavior. The suite also verifies that generic directory setup cannot recreate a catalog below a replaced ancestor and that a warn-mode policy discovery failure remains empty after the directory recovers, while successful discovery still loads policies and fail mode still rejects incomplete discovery. These deterministic tests verify ordering, object binding, and response ownership; they do not run a deliberately stalled mounted service. Hosted checks for the submitted head are reported by GitHub separately.Documentation and rollback
The English and Chinese guides now describe one
/v1/templatescatalog, its[template]configuration, import and lookup semantics, independent bounded list and item responses, capacity and publication rules, recovery behavior, and the explicit boundary with future sandbox creation.Omitting
template.import_rootdisables new imports. Reverting this PR restores the previous process-local routes; already published files remain on disk but are no longer exposed by the daemon.