docs(data-flow): propose silver->gold data contract (RFC)

[mitasovr]

What

RFC / proposal only — nothing is implemented by this PR. This adds a draft data contract for the planned database split: ingestion DB (bronze + staging + silver) vs gold DB, with a fixed interface between them.

New doc: docs/domain/ingestion-data-flow/specs/SILVER-GOLD-DATA-CONTRACT.md

Shape of the contract

Five datasets + two auxiliary dimensions, all based on the current silver tables (real column names and types):

Dataset	Silver table
commits (+ repository_branches dim)	class_git_commits / class_git_repository_branches
pull_requests	class_git_pull_requests
pull_request_comments	class_git_pull_requests_comments
task_field_history (+ task_statuses dim)	class_task_field_history / class_task_statuses
chat_activity	class_collab_chat_activity (daily aggregates; per-message is a future extension)

Each dataset section carries a JSON Schema, field descriptions, example records, and a growing list of Cases — normative interpretation rules for consumers, e.g.:

• patch_id on commits: all-branches stats use the first commit of each patch_id group; default-branch-only stats use the last one.
• Task close detection: by status_category = 'done' via the task_statuses dimension, never by status display names (same semantics as fix(task-delivery): detect closed tasks by statusCategory, not display name (#1541) #1732).
• Enriched events vs raw deltas: what the enrich step adds (running state, synthetic_initial creation baselines, deterministic ordering) and when to use which part.
• Honest NULLs in vendor chat counters (NULL = not reported, never coalesce to 0).

Normative additions over current state

Only two: person_id (stable UUID from Identity Resolution, resolved on the producer side) on every dataset, and patch_id on commits. Everything else the contract demands but silver doesn't deliver yet is listed explicitly in Appendix A (no git identity_inputs, no tenant column in task_field_history, GitLab diff stats = 0, no per-message data, …) — that list is effectively the implementation backlog if the format is accepted.

Feedback wanted

• Dataset granularity and naming
• The Cases format (Question / Rule / SQL sketch) as the place where interpretation rules accumulate
• Whether envelope naming should be harmonized (source_id vs insight_source_id, missing tenant in task history) as part of the split

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

@mitasovr, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 52 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ccbfd5c6-b6bb-4fce-a681-68729b94f63e

📥 Commits

Reviewing files that changed from the base of the PR and between 5124c78 and 7c9a831.

📒 Files selected for processing (1)

• docs/domain/ingestion-data-flow/specs/SILVER-GOLD-DATA-CONTRACT.md

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}