Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
92ebdef
feat: deterministic manifest schema v2
Fieldnote-Echo Jul 6, 2026
05fc05e
test: determinism merge gates for manifest bytes
Fieldnote-Echo Jul 6, 2026
92c6b24
fix: enforce canonical manifest paths at create and verify
Fieldnote-Echo Jul 6, 2026
0f7d4ba
ci: run ordvec-manifest tests across the OS matrix
Fieldnote-Echo Jul 6, 2026
55cb479
docs: schema v2 README refresh and changelog entry
Fieldnote-Echo Jul 6, 2026
d70b72f
fix: close backslash-absolute manifest path bypass
Fieldnote-Echo Jul 6, 2026
ef49e00
fix: drop stale active_manifest schema on sqlite registry init
Fieldnote-Echo Jul 6, 2026
c30bcf3
chore(release): bump lockstep version to 0.7.0 for manifest schema v2
Fieldnote-Echo Jul 14, 2026
22bd683
fix: platform-proof the schema-v2 test suite (windows CRLF golden, ma…
Fieldnote-Echo Jul 14, 2026
5b671d2
fix: portable missing-file assertion for the windows io error wording
Fieldnote-Echo Jul 14, 2026
d3d332f
Merge branch 'main' into feat/deterministic-manifest
Fieldnote-Echo Jul 14, 2026
2640224
fix: atomic sqlite migration, load-time schema check, audit doc fixes
Fieldnote-Echo Jul 14, 2026
903baa4
feat: typed verification codes + shared hash helpers (#288)
Fieldnote-Echo Jul 14, 2026
c2a8941
Merge branch 'main' into feat/deterministic-manifest
Fieldnote-Echo Jul 14, 2026
01fde71
chore: reset last-push attribution to branch owner
Fieldnote-Echo Jul 14, 2026
91cc852
docs: note serde_json feature-unification risk on the manifest conten…
Fieldnote-Echo Jul 14, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,11 @@ jobs:
run: cargo test --features test-utils
- name: cargo test (no default features)
run: cargo test --no-default-features
# The manifest crate embeds and resolves bundle paths, so its path
# handling is OS-sensitive; the dedicated ubuntu lane covers its
# feature matrix while this step covers the other OSes.
- name: cargo test -p ordvec-manifest
run: cargo test -p ordvec-manifest
- name: cargo build --release --features bench-utils --example bench_rank
run: cargo build --release --features bench-utils --example bench_rank

Expand Down
50 changes: 50 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,56 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

_No unreleased changes._

## 0.7.0 - 2026-07-13

### Added

- **`ordvec-manifest`: typed verification classification.** Every
verification issue code is now a named `pub const` in a `codes` module
(zero bare literals at emit sites), with a `#[non_exhaustive]`
`VerificationCode` enum and `ReportIssue::classification()` so
downstream integrity handling can branch on typed values instead of
comparing strings. A missing primary artifact or row-identity file is
reported with a dedicated `artifact_missing` / `row_identity_missing`
code (classified `ArtifactMissing` / `RowIdentityMissing`) only when the
file is genuinely absent; permission or I/O failures keep the generic
`*_path_unavailable` code and classify as `Unknown`, so a consumer never
mistakes an unreadable file for a missing one. `ReportIssue` gains optional structured mismatch
detail — artifact name plus expected/actual SHA-256 and sizes — at the
artifact, auxiliary, and row-identity mismatch sites, for lossless
downstream error construction. Reports without the new detail
serialize byte-identically to before (regression-tested).
- **`ordvec-manifest`: shared hash helpers.** New `sha256_bytes` and
bounded `sha256_reader` share `sha256_file_bounded`'s bounded/EINTR
read core; the sqlite registry's private duplicate hasher is deduped
onto the public helper.

### Changed

- **BREAKING (`ordvec-manifest`): deterministic manifest schema v2.** The
manifest schema version is now `ordvec.index_manifest.v2`. `manifest_id`
and `created_at` are removed from `IndexManifest`, creation omits the
optional `build` field (serialized as absent, not `null`), and auxiliary
artifact entries are sorted by
`(name, path)`, so identical bundle content serializes to byte-identical
manifests and `sha256(manifest.json)` is the bundle's content address.
Existing v1 manifests no longer parse; loading one fails with an error
naming both schema versions (zero back-compat, pre-release). Embedded
paths must now be canonical — bundle-relative, forward slashes, no `.`,
`..`, or empty segments — enforced both at creation (non-embeddable
inputs fail `create` instead of minting a manifest that fails its own
verification) and at verification (`*_path_not_canonical` codes, now
also covering calibration and encoder-distortion profile refs). Absolute
paths and escaping `..` paths remain available behind the existing
`allow_absolute_paths` / `allow_path_escape` opt-ins. The
`write_manifest_file` serialization form is documented as the single
canonical byte form: hashing and signing operate on stored bytes, and any
serializer change is a schema-version event. The sqlite report registry
drops its `manifest_id` column: the cached `verification_reports` table is
migrated in place on open (rows preserved, under one atomic transaction),
while the rebuildable `active_manifest` pointer is reset and must be
re-activated.

## 0.6.0 - 2026-07-04

### Performance
Expand Down
10 changes: 5 additions & 5 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "ordvec"
version = "0.6.0"
version = "0.7.0"
edition = "2021"
rust-version = "1.89" # AVX-512 intrinsics stabilized in 1.89.0; also clears the 1.87 floor from u64::is_multiple_of
description = "Training-free ordinal & sign quantization for vector retrieval"
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@ Details in [`docs/RANK_MODES.md`](docs/RANK_MODES.md).

```toml
[dependencies]
ordvec = "0.6"
ordvec = "0.7"

# Or, to track unreleased `main`, use a git dependency instead:
# ordvec = { git = "https://github.com/Project-Navi/ordvec" }
Expand Down
2 changes: 1 addition & 1 deletion THREAT_MODEL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Threat Model — `ordvec`

> **Status:** v0.6.0 (pre-1.0), 2026-06-15. This is the maintained threat model
> **Status:** v0.7.0 (pre-1.0), 2026-06-15. This is the maintained threat model
> for the `ordvec` Rust crate, C ABI, Go wrapper, PyO3/maturin Python bindings,
> and the `ordvec-manifest` sidecar verifier. It is reviewed when the
> attack surface changes (new persistence formats, new `unsafe` kernels, new
Expand Down
2 changes: 1 addition & 1 deletion fuzz/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion ordvec-ffi/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "ordvec-ffi"
version = "0.6.0"
version = "0.7.0"
edition = "2021"
rust-version = "1.89"
publish = false
Expand Down
2 changes: 1 addition & 1 deletion ordvec-manifest-python/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "ordvec-manifest-python"
version = "0.6.0"
version = "0.7.0"
edition = "2021"
rust-version = "1.89"
description = "Python bindings for ordvec-manifest index provenance verification"
Expand Down
2 changes: 1 addition & 1 deletion ordvec-manifest-python/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "maturin"

[project]
name = "ordvec-manifest"
version = "0.6.0"
version = "0.7.0"
description = "Python bindings for ordvec index manifest verification"
readme = "README.md"
requires-python = ">=3.10"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,4 +50,4 @@
"create_manifest",
]

__version__ = "0.6.0"
__version__ = "0.7.0"
2 changes: 0 additions & 2 deletions ordvec-manifest-python/tests/test_manifest_bindings.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,6 @@ def write_unloadable_manifest(tmp_path):
digest = hashlib.sha256(artifact.read_bytes()).hexdigest()
manifest = {
"schema_version": ordvec_manifest.SCHEMA_VERSION,
"manifest_id": "urn:uuid:7c66ad6e-bdde-49a8-b420-f1136d04f5bd",
"created_at": "2026-06-09T00:00:00Z",
"artifact": {
"path": artifact.name,
"sha256": digest,
Expand Down
4 changes: 2 additions & 2 deletions ordvec-manifest/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "ordvec-manifest"
version = "0.6.0"
version = "0.7.0"
edition = "2021"
rust-version = "1.89"
license = "MIT OR Apache-2.0"
Expand Down Expand Up @@ -29,7 +29,7 @@ required-features = ["cli"]
chrono = { version = "0.4.44", default-features = false, features = ["clock", "std"] }
clap = { version = "4.6.1", features = ["derive"], optional = true }
hex = "0.4.3"
ordvec = { version = "0.6.0", path = ".." }
ordvec = { version = "0.7.0", path = ".." }
rusqlite = { version = "0.40.0", optional = true }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
Expand Down
14 changes: 9 additions & 5 deletions ordvec-manifest/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,12 @@ ordvec-manifest verify --manifest path/to/index.manifest.json
From a workspace checkout, prefix the same commands with
`cargo run -p ordvec-manifest --`.

The schema version is `ordvec.index_manifest.v1`. Relative paths resolve from
The schema version is `ordvec.index_manifest.v2`. The v2 schema is
deterministic: identical bundle content serializes to identical manifest
bytes, so `sha256(manifest.json)` is the bundle's content address. Manifests
carry no `manifest_id` or `created_at`, auxiliary artifact entries are sorted
by `(name, path)`, and embedded paths must be canonical (bundle-relative,
forward slashes, no `.`, `..`, or empty segments). Relative paths resolve from
the manifest file's directory, absolute paths are rejected by default, and
relative paths may not escape the manifest directory unless explicitly allowed.
`create` follows the same policy: by default it emits only paths that should
Expand Down Expand Up @@ -229,11 +234,11 @@ A consuming database can keep the ordvec row identity as
`RowIdentity::RowIdIdentity { row_count }` and declare its ID sidecar file as a
required auxiliary artifact (e.g. `app.ids`). That makes the vector row count an
ordvec invariant while leaving the caller's `u64` document IDs as caller-owned
sidecar bytes. Do not encode the ID sidecar as `RowIdentity::Jsonl`: v1 JSONL
sidecar bytes. Do not encode the ID sidecar as `RowIdentity::Jsonl`: JSONL
row identity is UUID-oriented (`id_kind = "uuid"`), and generic row-map ID
formats are intentionally deferred to
[#145](https://github.com/Project-Navi/ordvec/issues/145). The reserved
`row_identity.db` metadata block is rejected in v1 because it is not byte-bound
`row_identity.db` metadata block is rejected because it is not byte-bound
or path-checked.

Stable row-identity boundary codes:
Expand Down Expand Up @@ -273,6 +278,7 @@ Stable sidecar states:
| `failed` | Code-specific | Path policy, hashing, size, digest, or limit validation failed. |

Common `failed` reason codes include `auxiliary_artifact_path_empty`,
`auxiliary_artifact_path_not_canonical`,
`auxiliary_artifact_base_dir_unavailable`,
`auxiliary_artifact_path_unavailable`,
`auxiliary_artifact_path_escape_rejected`,
Expand All @@ -287,7 +293,6 @@ and records checks that were intentionally not run, such as
{
"ok": true,
"checked_at": "2026-06-03T17:20:00Z",
"manifest_id": "urn:uuid:11111111-1111-4111-8111-111111111111",
"artifact": {
"manifest_path": "index.ovrq",
"observed_path": "index.ovrq",
Expand Down Expand Up @@ -346,7 +351,6 @@ read and absent when the file is missing:
{
"ok": false,
"checked_at": "2026-06-03T17:21:00Z",
"manifest_id": "urn:uuid:11111111-1111-4111-8111-111111111111",
"artifact": {
"manifest_path": "index.ovrq",
"observed_path": "index.ovrq",
Expand Down
Loading
Loading