Skip to content

fix(file-storage): Postgres jsonb binding, S3 readiness probe, standalone doc corrections#4219

Open
ffedoroff wants to merge 2 commits into
constructorfabric:mainfrom
ffedoroff:fix/file-storage-pg-jsonb-s3-readiness
Open

fix(file-storage): Postgres jsonb binding, S3 readiness probe, standalone doc corrections#4219
ffedoroff wants to merge 2 commits into
constructorfabric:mainfrom
ffedoroff:fix/file-storage-pg-jsonb-s3-readiness

Conversation

@ffedoroff

@ffedoroff ffedoroff commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Independent fixes split out of #4218 (the file-storage audit-remediation PR) to keep both PRs under the 50-file review limit. This PR is fully standalone — it does not depend on #4218 and can merge in any order relative to it.

Postgres jsonb compatibility (HIGH)

policies.body / retention_rules.body SeaORM entities declared Text/String while the Postgres DDL declares jsonb — binding a text parameter into a jsonb column fails on Postgres ("column is of type jsonb but expression is of type text"). The test suite runs SQLite only, so every policy/retention write on the documented production target was unexercised and broken.

  • Entities switched to column_type = "Json" (same pattern as the audit/events outbox entities); repos bind serde_json::Value via to_value/from_value.
  • Regression tests assert the entity column types stay Json.

S3 readiness probe false positive

S3Backend::is_ready probed a well-known key with HeadObject; a HEAD 404 carries no body, so NoSuchBucket was indistinguishable from NoSuchKey and a missing/misconfigured bucket reported ready while all real I/O failed. Now probes the bucket itself via ListObjectsV2 (max-keys=1). Regression test included.

Standalone doc corrections (describe current behavior, no code dependency)

  • ADR-0006: only the multipart full re-read was eliminated — the single-part finalize read-back is retained, and complete still issues an 8 KiB ranged read for MIME sniffing.
  • content-hash-modes.md: part_count-vs-manifest cross-check marked as a known gap (not implemented); the client re-verification flow now says the manifest must be retained from the complete response (no metadata endpoint returns it).
  • infra module doc: lists the authz/metrics submodules and the S3 backend.
  • SDK README: honest note that the inter-gear client trait (FileStorageClientV1) is still a stub.

Test plan

  • cargo test -p cf-gears-file-storage on this branch alone — 361 passed, 0 failed
  • cargo clippy --all-targets clean, cargo fmt --check clean
  • cfs validate — 0 errors
  • New regression tests: entity column types are Json; is_ready errors against a missing bucket

https://claude.ai/code/session_01WZUBxb7myjUZHMYmbRtJqG

Summary by CodeRabbit

  • Documentation
    • Clarified multipart content-hash verification semantics, including where manifest is available and updated client re-verification requirements.
    • Refined assertions around what multipart completion does (hashing vs. MIME sniffing) and noted a documented gap in manifest/part-count cross-checking.
    • Updated file-storage API README to reflect current public surface status.
  • Bug Fixes
    • Improved S3 backend readiness probing to correctly treat reachable buckets as ready and missing buckets as errors.
    • Corrected JSON persistence for policy and retention-rule bodies by aligning stored column types and serialization/deserialization.
  • Tests
    • Added/updated regression coverage for S3 readiness and JSON column typing behavior.

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@ffedoroff, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 26 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: a19b69c3-02eb-4e43-a47e-a4e4ebbb7a81

📥 Commits

Reviewing files that changed from the base of the PR and between 74acc63 and b0477c1.

📒 Files selected for processing (1)
  • gears/file-storage/docs/features/content-hash-modes.md
📝 Walkthrough

Walkthrough

The PR updates content-hash documentation, changes S3 readiness checks to bucket listing, aligns policy and retention persistence with JSON columns, and documents the current file-storage API and infrastructure adapters.

Changes

Content-hash documentation

Layer / File(s) Summary
Hashing and manifest verification semantics
gears/file-storage/docs/ADR/..., gears/file-storage/docs/features/content-hash-modes.md
Documents multipart on-the-fly hashing, retained manifest requirements, the part-count verification gap, and bounded MIME-sniffing reads.

S3 readiness probing

Layer / File(s) Summary
Bucket-scoped readiness check
gears/file-storage/file-storage/src/infra/backend/s3.rs, gears/file-storage/file-storage/src/infra/backend/s3_tests.rs
is_ready now uses bucket-scoped ListObjectsV2; tests cover empty existing buckets and missing buckets.

JSON policy and retention persistence

Layer / File(s) Summary
JSON column and repository mapping
gears/file-storage/file-storage/src/infra/storage/entity/{policy,retention_rule}.rs, gears/file-storage/file-storage/src/infra/storage/repo/{policy_repo,retention_rule_repo}.rs
Entity bodies use SeaORM JSON columns, with repository serialization and deserialization updated to JSON values and column-type regression tests.

File-storage API and infrastructure documentation

Layer / File(s) Summary
Public API and adapter status
gears/file-storage/file-storage-sdk/README.md, gears/file-storage/file-storage/src/infra/mod.rs
Documents the current client trait status and adds authorization and metrics adapters to the infrastructure description.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Suggested reviewers: artifizer

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main changes: Postgres jsonb binding fixes, the S3 readiness probe update, and documentation corrections.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@gears/file-storage/docs/features/content-hash-modes.md`:
- Around line 122-126: Update the `p2` client metadata step to refer to the
serialized `VersionDto` field `hash` instead of `hash_value`, while preserving
the existing `hash_mode`, `part_count`, and manifest availability guidance.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: fee42fb8-2b1e-4457-af52-f84efe1e0127

📥 Commits

Reviewing files that changed from the base of the PR and between e7361e7 and 1c422d1.

📒 Files selected for processing (10)
  • gears/file-storage/docs/ADR/0006-cpt-cf-file-storage-adr-content-hash-modes.md
  • gears/file-storage/docs/features/content-hash-modes.md
  • gears/file-storage/file-storage-sdk/README.md
  • gears/file-storage/file-storage/src/infra/backend/s3.rs
  • gears/file-storage/file-storage/src/infra/backend/s3_tests.rs
  • gears/file-storage/file-storage/src/infra/mod.rs
  • gears/file-storage/file-storage/src/infra/storage/entity/policy.rs
  • gears/file-storage/file-storage/src/infra/storage/entity/retention_rule.rs
  • gears/file-storage/file-storage/src/infra/storage/repo/policy_repo.rs
  • gears/file-storage/file-storage/src/infra/storage/repo/retention_rule_repo.rs

Comment thread gears/file-storage/docs/features/content-hash-modes.md Outdated
@ffedoroff
ffedoroff force-pushed the fix/file-storage-pg-jsonb-s3-readiness branch 2 times, most recently from 88e7da2 to 1bda67c Compare July 16, 2026 17:42
…lone doc corrections

Independent fixes split out of the file-storage audit-remediation PR to keep
each PR reviewable:

- policies.body / retention_rules.body entities switched from Text/String to
  column_type = "Json", matching the Postgres jsonb DDL (same pattern as the
  audit/events outbox entities); repos bind serde_json::Value instead of a
  serialized string. Writes previously failed on the documented Postgres
  target ("column is of type jsonb but expression is of type text") while
  passing silently under the SQLite-only test suite. Regression tests pin
  the entity column types.
- S3Backend::is_ready now probes the bucket itself (ListObjectsV2,
  max-keys=1) instead of a HeadObject on a probe key: a HEAD 404 cannot
  distinguish NoSuchBucket from NoSuchKey, so a missing or misconfigured
  bucket previously passed /readyz while every real read/write failed.
- ADR-0006 correction: only the multipart full re-read was eliminated; the
  single-part finalize read-back is retained, and complete still issues an
  8 KiB ranged read for MIME sniffing.
- content-hash-modes.md: the part_count-vs-manifest cross-check is marked as
  a known gap (not implemented), and the client re-verification flow now
  says the manifest must be retained from the complete response (no
  metadata endpoint returns it).
- infra module doc lists the authz and metrics submodules and S3 backend;
  SDK README states the inter-gear client trait is still a stub.

Claude-Session: https://claude.ai/code/session_01WZUBxb7myjUZHMYmbRtJqG
Signed-off-by: Roland From <rfedorov@linkentools.com>
@ffedoroff
ffedoroff force-pushed the fix/file-storage-pg-jsonb-s3-readiness branch from 1bda67c to 74acc63 Compare July 16, 2026 18:42

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@gears/file-storage/docs/features/content-hash-modes.md`:
- Line 735: Update the multipart client-side re-verification guidance in the
affected table to reference the POST .../complete response field content_hash
instead of root, while retaining manifest as documented and preserving the
existing verification steps.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 12029fad-274b-423e-a1be-82011a527b9a

📥 Commits

Reviewing files that changed from the base of the PR and between 1c422d1 and 74acc63.

📒 Files selected for processing (10)
  • gears/file-storage/docs/ADR/0006-cpt-cf-file-storage-adr-content-hash-modes.md
  • gears/file-storage/docs/features/content-hash-modes.md
  • gears/file-storage/file-storage-sdk/README.md
  • gears/file-storage/file-storage/src/infra/backend/s3.rs
  • gears/file-storage/file-storage/src/infra/backend/s3_tests.rs
  • gears/file-storage/file-storage/src/infra/mod.rs
  • gears/file-storage/file-storage/src/infra/storage/entity/policy.rs
  • gears/file-storage/file-storage/src/infra/storage/entity/retention_rule.rs
  • gears/file-storage/file-storage/src/infra/storage/repo/policy_repo.rs
  • gears/file-storage/file-storage/src/infra/storage/repo/retention_rule_repo.rs
🚧 Files skipped from review as they are similar to previous changes (6)
  • gears/file-storage/file-storage/src/infra/storage/entity/retention_rule.rs
  • gears/file-storage/file-storage/src/infra/mod.rs
  • gears/file-storage/file-storage/src/infra/storage/repo/policy_repo.rs
  • gears/file-storage/file-storage/src/infra/storage/entity/policy.rs
  • gears/file-storage/docs/ADR/0006-cpt-cf-file-storage-adr-content-hash-modes.md
  • gears/file-storage/file-storage/src/infra/storage/repo/retention_rule_repo.rs

Comment thread gears/file-storage/docs/features/content-hash-modes.md Outdated
CodeRabbit review: the client-facing steps referenced hash_value (the DB
column) and root, but the wire fields are VersionDto.hash and the complete
response's content_hash. DB-context references keep hash_value.

Claude-Session: https://claude.ai/code/session_01WZUBxb7myjUZHMYmbRtJqG
Signed-off-by: Roland From <rfedorov@linkentools.com>
@ffedoroff
ffedoroff force-pushed the fix/file-storage-pg-jsonb-s3-readiness branch from f7d08a9 to b0477c1 Compare July 16, 2026 20:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant