Skip to content

Repository files navigation

structured-rag-dotnet

ci .NET 10 licence: MIT

Query a database by meaning rather than by SQL. Rows are turned into sentences, embedded, and retrieved semantically — so a question can find a record without sharing a single word with it.

New to this? GETTING-STARTED.md builds the whole thing from an empty directory, and is honest about where the approach breaks.

$ srag search "which buyers purchase in bulk for their business"
Filter: kind=Entity

[1] customer:2  Entity  score=0.254
    Tom Okonkwo is a customer from Nigeria who signed up on 23 November 2024. They have
    placed 4 completed orders worth 1,760.00 in total. They buy Equipment and Wholesale.

Neither "bulk" nor "business" appears anywhere in that database — not in the schema, not in a single row. Vector search found the wholesale customer anyway. That is what this approach is for.

(In fairness: the runner-up scored 0.248, and buys only Coffee and Supplies. The margin is thin. See Limitations.)


Read this first: it is not a replacement for SQL

This repository exists alongside hello-text2sql-dotnet, which answers the same questions by generating SQL. They are complementary, and pretending otherwise would waste your time.

Structured RAG (this) Text-to-SQL
"Which customers seem to buy in bulk?" Good — semantic match Poor — no keyword to match on
"Find orders like this one" Good — that is what vectors do Awkward
"Total revenue last month" Works only if pre-computed Exact, always
"Average order value" Fails — see below Exact
Freshness Stale until re-indexed Always current
Cost per question One embedding call One LLM call
Setup cost Index every row None

Retrieval cannot add up. That single sentence is most of what you need to know. Ask for a total and retrieval returns the handful of records that best match the question — not all of them.


The failure, demonstrated

Asked for something with no pre-computed rollup:

$ srag ask "what is the average order value across the whole database"

A: To calculate the average order value, we sum up all revenues and divide by the
   number of completed orders that are counted (excluding refunds). From facts
   [1], [2], [3], [5], and excluding fact [6]'s refunded order:

   Total revenue = 493.00 + 2,358.00 + 545.50 + 940.00 = 4,336.50
   Number of completed orders counted = 1+5+3+1 = 10
   Average order value is therefore: 4,336.50 / 10 = 433.65

The true answer is 284.75 (4,840.80 / 17 — verified directly against SQL).

Look closely at what went wrong. The arithmetic is correct. 4,336.50 / 10 really is 433.65. The model did not miscalculate — it calculated perfectly over the wrong set of numbers, because retrieval handed it four months out of nine. It had no way to know the other five existed.

Worse, it is not even reliably wrong. An earlier run of the same question produced 548.83, having retrieved a different subset and stated its own assumptions along the way. Same question, same data, a different confident number each time.

The system prompt already says "Never add up separate facts yourself: the facts shown are only the ones that matched, not all of them." It did it anyway — the usual lesson: prompts are requests, not constraints.

So the tool detects the shape of the problem and says so:

Warning: this looks like a question that needs a total, but nothing
pre-computed was retrieved. Retrieval returns the few records that matched,
not all of them, so any number above may have been derived from a partial set.
Check the facts below, or use SQL for exact aggregates.

That is a mitigation, not a fix. For exact aggregates, use SQL.

What does work

Because the indexer pre-computes monthly rollups, this is a lookup rather than a calculation:

$ srag ask "what was the total revenue last month"
Filter: kind=Aggregate  date 2026-07-01 to 2026-08-01

A: According to fact [1], the total revenue last month (July 2026) was $2,358.00
   across completed orders after accounting for a refunded order that is not
   included in this figure.

2,358.00 matches the hand-written SQL exactly. Pre-computing turns an aggregation into a lookup — but only for the aggregates somebody thought to pre-compute.

(One quirk worth noticing: the model wrote $2,358.00. There is no currency symbol anywhere in the data. Even when restating a retrieved fact correctly, it adds detail that was never there.)


The technique: verbalisation

This is the part that makes structured RAG work, and the step most people skip.

A database row is close to meaningless to an embedding model:

id=11, customer_id=1, placed_on=2026-07-04, status=completed

customer_id 1 carries no semantic content. Two orders from completely different customers embed almost identically. Ask "which wholesale customers in India ordered recently" and no vector built from that row can help.

Verbalised with its joined context, the same row becomes searchable:

Order 11 was placed by Priya Raman from India on 4 July 2026 and is completed.
It contains 25 x Wholesale Meridian 1kg and 4 x Descaling Tablets x20,
totalling 1,273.00. Product categories: Supplies and Wholesale.

Every noun is now something a question can match against. The price is that indexing has to do the joins SQL would have done at query time — work moves from query time to index time, and the index goes stale when the data changes.

Three kinds of document

Kind One per Answers
Row Order, joined with customer and products "Which orders included a grinder?"
Entity Customer, aggregated across all orders "Who are our biggest customers?"
Aggregate Month, pre-summed "What was revenue in July?"

Entity documents exist because a single order tells you nothing about a customer's overall value. Aggregate documents exist because retrieval cannot add up.


Quick start

Prerequisites: the .NET 10 SDK. Ollama is optional.

git clone https://github.com/kuldeepcodes/structured-rag-dotnet.git
cd structured-rag-dotnet

dotnet build -c Release
dotnet run -c Release --project tests/StructuredRag.Tests    # 72 tests

dotnet run --project src/StructuredRag.Cli -- seed     # build the sample database
dotnet run --project src/StructuredRag.Cli -- index    # verbalise and embed
dotnet run --project src/StructuredRag.Cli -- ask "who buys wholesale"

For real semantic search and generated answers:

ollama pull all-minilm    # ~45 MB, embeddings
ollama pull phi3          # ~2.2 GB, generation

Without them it still runs. A deterministic hashing embedder keeps the pipeline working with no downloads — retrieval quality drops to keyword matching, but every part is real, and that is how the tests and CI run.


Setting up the database

Sample data lives in db/schema.sql — a plain script you can read or run yourself:

sqlite3 shop.db < db/schema.sql          # any SQLite client
srag seed                                # or let the tool do it

Dates are generated relative to date('now'), so "last month" keeps working long after this was written.


Commands

Command What it does
srag seed Build the sample database from db/schema.sql
srag index Verbalise every row, embed it, write the index
srag ask "<question>" Retrieve facts and answer
srag search "<question>" Show what retrieval found, without generating
srag show [filter] Print the verbalised documents themselves
srag stats Describe the index

srag show is the debugging command. When a query misses, the reason is almost always visible in the sentence that was generated for the row.


How it works

flowchart TD
    subgraph INDEX["INDEX &mdash; once per data change"]
        direction LR
        DB[("SQLite")]
        JOIN["<b>join</b><br/>orders + customers<br/>+ products"]
        VERB["<b>verbalise</b><br/>row &rarr; sentence<br/>ids &rarr; names"]
        EMB["<b>embed</b><br/>one batched call"]
        DB --> JOIN --> VERB --> EMB
    end

    STORE[("<b>index.json</b><br/>text + vector + metadata")]
    EMB --> STORE

    subgraph ASK["ASK &mdash; per question"]
        direction LR
        Q["Question"]
        INF["<b>infer filter</b><br/>dates, status, kind"]
        FILT["<b>filter</b><br/>exact, structural"]
        RANK["<b>rank</b><br/>cosine similarity"]
        Q --> INF --> FILT --> RANK
    end

    STORE --> ASK
    RANK --> LLM["LLM restates the facts"]
    LLM --> ANS["<b>Answer</b> + fact ids"]
    RANK -.->|aggregation question,<br/>no single rollup| WARN["warn: number may<br/>be derived from a<br/>partial set"]

    classDef store fill:#0d3b66,stroke:#0d3b66,color:#fff
    classDef good fill:#1b5e20,stroke:#1b5e20,color:#fff
    classDef stop fill:#7f1d1d,stroke:#7f1d1d,color:#fff

    class STORE store
    class ANS good
    class WARN stop
Loading

What happens when you ask a question

sequenceDiagram
    autonumber
    actor U as You
    participant CLI as Program
    participant RQ as RagQuery
    participant ST as DocumentStore
    participant EM as IEmbedder
    participant M as LLM (phi3)

    U->>CLI: srag ask "total revenue last month"
    CLI->>ST: DocumentStore.Open(index.json)
    ST-->>CLI: documents + vectors + metadata

    CLI->>RQ: Ask(question)
    RQ->>RQ: InferFilter(question)
    Note over RQ: "last month" is a constraint, not a<br/>similarity. Calendar boundaries, exact.<br/>"revenue" steers to kind=Aggregate.

    RQ->>EM: EmbedOne(question)
    EM-->>RQ: vector

    RQ->>ST: Search(vector, filter, topK)
    Note over ST: Filter first, exactly. Then rank<br/>what survives. The other order<br/>can leave you with nothing.
    ST-->>RQ: ranked facts

    RQ->>RQ: HasAggregationRisk(question, facts)
    Note over RQ: One rollup = a fact to read.<br/>Zero or many = a sum to derive,<br/>which is where it invents numbers.

    RQ->>M: system rules + numbered facts
    M-->>RQ: "Revenue was 2,358.00 [1]."
    RQ-->>U: answer, fact ids, and a warning if at risk
Loading

Four decisions worth knowing about

Filters run before ranking. Filtering afterwards means your top five might all be discarded, leaving nothing. Dates are stored ISO-formatted so string comparison is date comparison — no parsing, no timezone ambiguity.

Dates are prose in the text, ISO in the metadata. "4 July 2026" embeds better than "2026-07-04"; the ISO form is what filtering needs. Both are kept.

Revenue questions are steered to the rollups. A total lives in a monthly summary, not in five individual orders. Without that steer, retrieval returns orders and the model tries arithmetic.

The inferred filter is printed. A surprising result is usually a surprising filter, and seeing date 2026-07-01 to 2026-08-01 explains an empty result immediately.


When to use which

Use structured RAG when questions are fuzzy, exploratory, or about similarity — "which customers look like they might churn", "find orders similar to this complaint".

Use text-to-SQL when you need an exact number — any total, any average, any count.

Serious systems run both and route between them. The routing question — does this need arithmetic, or judgement? — is the interesting engineering, and this repository deliberately leaves it as an exercise rather than pretending one approach covers everything.


Tests

dotnet run -c Release --project tests/StructuredRag.Tests    # 72 tests

They cover verbalisation (names not ids, categories stated, grammar for singular and plural), filtering (date ranges exclusive at the end, conjunctive combination, documents with no date), persistence, and the aggregation-risk detector in both directions.

Deterministic throughout — hashing embedder, stub chat model — so no network, no model, no GPU.


Limitations

  • Aggregation. Covered above. The honest answer is "use SQL".
  • Similar entities cluster. Customer documents share a sentence structure, so they embed close together and fine distinctions between them get blurred. In the wholesale example the runner-up (0.248) buys no wholesale at all, against a winner at 0.254. Ranking is real but the margin is thin; treat entity search as a shortlist, not an answer.
  • Staleness. The index is a snapshot. Re-run srag index after the data changes.
  • Index cost. Every row is embedded. A million rows is a million embeddings.
  • Schema-specific verbalisation. Verbaliser knows about orders and customers. Another schema needs its own, and generating that automatically is an open problem.
  • Exhaustive search. Exact, and fine to roughly a hundred thousand documents; past that you want an ANN index.

Related

Licence

MIT

About

Query a database by meaning, not SQL: rows become sentences, sentences become vectors. With an honest look at where retrieval cannot add up.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages