Golem

A compiled customer service bot. The LLM acts as a compiler: it runs once (or a few times) to build a self-contained knowledge bundle. At runtime, a deterministic CPU-only engine reads the bundle — no external calls, no generative model, no variable latency.

Design

See docs/design-spec.md for the original design document, docs/design-analysis.md for a critical review with recommendations, and docs/runtime-model-decision.md for the measured decision on what the runtime is allowed to load.

The runtime is numpy + scipy + scikit-learn + tokenizers. No torch, no transformers, no onnxruntime.

Setup

Requires Python 3.13+ and uv.

uv sync --all-extras

golem is then on the venv's PATH. To use it from anywhere:

uv tool install --editable '.[compile]'    # puts `golem` in ~/.local/bin

Try it

One script builds a three-intent Spanish bundle and asks it questions it has never seen. It is the real CLI in the real order — read it to see the whole protocol an agent follows.

./examples/acme/seed.sh /tmp/acme

The first run downloads and converts the shared embedding table (~440 MB, once per machine). Afterwards:

golem ask "no me llega lo que compre" --path /tmp/acme
golem status --path /tmp/acme

Expect roughly: unseen paraphrases answered, vague questions clarified, out-of-domain questions escalated — and a compile report stating how much of what it cannot answer it actually catches.

Two ways to use it

As a server. golem serve holds one embedding table and any number of bundles, and answers over HTTP. See below.

As a library, in front of a bot you already have. Golem answers what it is sure about; your existing LLM or RAG path takes everything else. Nothing is replaced, so nothing regresses — and the turn log records which questions the fallback had to take, which is the work-list for the next bundle.

frontline = Frontline(
    engine=Engine.load("dist", encoder.load()),
    fallback=my_existing_rag,          # returns a string, or None to decline
    sink=my_database_sink,             # one method, one argument
)
handled = frontline.ask(text)
handled.text          # what to say
handled.from_bundle   # who answered
handled.reply.options # candidates, for inline buttons

A Reply carries no composed message on purpose: golem decides, the host presents. engine.render(reply) composes a default for hosts that just want a string. Worked example: examples/frontline/bot.py.

Building a bundle

Golem never calls an LLM. You are the compiler: read the client's documentation, then record what you find.

golem init acme-support
golem intent add order_status --description "..." --label "el estado de tu pedido"
echo '["donde esta mi pedido","no me ha llegado"]' | golem variant load order_status
golem response set order_status "Puedes seguir tu pedido con el codigo del correo."
echo '["trabajan los domingos","que tiempo hara manana"]' | golem negative load
golem status          # what is still missing
golem compile         # fit, calibrate, emit

Never edit golem.json by hand. Every command ends by telling you the next step, so golem status is the only thing you need to remember.

The negatives are not optional: the thresholds that decide when the bot escalates are solved against them, and a project without enough of them does not compile.

License

MIT