Skip to content

Persist raw items/inferences in SQLite (keep JSON fallback) - #6

Open
brycejohnson1417 wants to merge 1 commit into
mainfrom
jarvis/sqlite-storage-and-schema
Open

Persist raw items/inferences in SQLite (keep JSON fallback)#6
brycejohnson1417 wants to merge 1 commit into
mainfrom
jarvis/sqlite-storage-and-schema

Conversation

@brycejohnson1417

Copy link
Copy Markdown
Owner

Summary

  • Add env-driven config (paths, batch size).
  • Store ingested raw items + generated inferences in a local SQLite DB for durability.
  • Keep legacy raw_data.json and inferences.json as fallback/compat; auto-import legacy inferences on startup.

Why

  • JSON files are fragile and hard to evolve; SQLite provides durable local state without external deps.

Notes

  • API response fields remain compatible with the existing frontend (id/source/content/inference/confidence).
  • This PR intentionally does not change the export security gate or model pin (handled separately).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant