Skip to content

[Client P1] Ingest the complete corpus for FRDC and GRDC (not a subset) #40

Description

@jaysanderson

Client feedback (priority 1)

"Ingesting the whole corpus (the same goes for GRDC). With a subset, even if the retrieval is good, people will say it's missing this and that and lose confidence." Also: "without the whole corpus ingested it's hard to say whether the retrieval and answers are good."

Status (2026-08-31)

  • FRDC: full corpus load IN PROGRESS via the resumable loader (~2,690 / 3,920 docs, finishing today). Top active workstream over the weekend.
  • GRDC: 1,070 resources loaded + fully merchandised. VERIFY: is 1,070 the complete GRDC corpus or a subset? Confirm against the source of truth and top up if partial.

Action

  • Finish FRDC load; run post-load enrichment + graph + taxonomy passes.
  • Confirm GRDC completeness; ingest the remainder if a subset.
  • Re-evaluate retrieval/answer quality once complete (client is reserving judgement until then).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions