Skip to content

feat: support local Ollama for embeddings and fact-extraction LLM - #79

Merged
imonroe merged 3 commits into
mainfrom
claude/github-issue-51-ollama
Jul 9, 2026
Merged

feat: support local Ollama for embeddings and fact-extraction LLM#79
imonroe merged 3 commits into
mainfrom
claude/github-issue-51-ollama

Conversation

@imonroe

@imonroe imonroe commented Jul 8, 2026

Copy link
Copy Markdown
Owner

Closes #51.

What

Adds first-class, opt-in config for running mem0's embedder and/or fact-extraction LLM against a local Ollama server — zero per-call cost and no content leaving the host — while keeping anthropic + openai as the default. As the issue notes, _build_config already abstracts the providers, so this is a config + wiring + docs change, not a re-architecture. No change to the single-Memory-per-process invariant.

Changes

  • app/config.py — new OLLAMA_BASE_URL setting (default http://localhost:11434). The existing _require_provider_keys validator already enforces a provider's key only when that provider is selected, so ollama needs no ANTHROPIC_API_KEY/OPENAI_API_KEY.
  • app/memory.py_provider_config() is now provider-aware: cloud providers get the API key injected explicitly (as before), while an ollama provider gets mem0's ollama_base_url key (and embedding_dims for the embedder) and no key. Wired through _build_config() for both llm and embedder, so fully-local and mixed cloud/local setups both work.
  • .env.example — documented OLLAMA_BASE_URL + a commented-out fully-local provider block.
  • docker-compose.yml — commented-out optional bundled ollama service (+ ollama_data volume) for the self-contained stack, with model-pull notes and an optional GPU block.
  • docs/USER_GUIDE.md — new "Running fully local with Ollama" section and config-table rows, prominently restating the critical MEM0_EMBED_DIMS-must-match invariant (switching embed models — including cloud↔local — means dropping/recreating the Qdrant collection). docs/DEVELOPER_GUIDE.md — updated the _provider_config internals note.

Config keys (verified against mem0 2.0.2)

{"llm":      {"provider": "ollama", "config": {"model": "llama3.1:8b", "ollama_base_url": "http://localhost:11434"}},
 "embedder": {"provider": "ollama", "config": {"model": "nomic-embed-text", "ollama_base_url": "http://localhost:11434", "embedding_dims": 768}}}

Verification

  • ruff check clean; 241 tests pass (pytest -q) against the pinned mem0ai==2.0.2.
  • New tests: Ollama config building (fully-local + mixed local-LLM/cloud-embed), OLLAMA_BASE_URL default/override, and — critically — the Ollama config is validated against the real mem0.configs.base.MemoryConfig schema, so a future mem0 rename of ollama_base_url is caught in CI rather than at deploy time.
  • Default behavior is unchanged: with no Ollama vars set, the config is byte-for-byte the same anthropic+openai shape as before (existing test_build_config_shape still passes untouched).

🤖 Generated with Claude Code


Generated by Claude Code

Adds opt-in config for running mem0's embedder and/or LLM against a local
Ollama server — zero per-call cost and no content leaving the host — while
keeping anthropic+openai as the default.

- config.py: new OLLAMA_BASE_URL setting (default http://localhost:11434).
  The existing provider-key validator already skips key checks for non
  anthropic/openai providers, so ollama needs no API key.
- memory.py: _provider_config() is now provider-aware — cloud providers get
  the API key injected; an "ollama" provider gets mem0's ollama_base_url (and
  embedding_dims for the embedder) and no key. Wired through _build_config for
  both llm and embedder, so fully-local and mixed cloud/local setups work.
- .env.example / docker-compose.yml: documented vars plus a commented-out
  optional bundled `ollama` service for the Compose stack.
- USER_GUIDE: "Running fully local with Ollama" section + config-table rows,
  prominently restating the MEM0_EMBED_DIMS-must-match invariant (recreate the
  Qdrant collection when switching embed models). DEVELOPER_GUIDE: updated the
  _provider_config internals note.
- Tests: ollama config building (fully-local + mixed), OLLAMA_BASE_URL
  default/override, and Ollama config validated against the real mem0
  MemoryConfig schema so a key rename is caught in CI.

Closes #51.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XQXUARh5h5hRFg67toenEX

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds opt-in support for running mem0’s embedder and/or fact-extraction LLM against a local Ollama server, while preserving the existing Anthropic (LLM) + OpenAI (embedder) defaults. This is primarily provider-config wiring plus documentation and tests to ensure the generated mem0 config matches the pinned mem0 schema.

Changes:

  • Add OLLAMA_BASE_URL setting and wire ollama_base_url (+ embedding_dims for embeddings) into mem0 provider config.
  • Expand docs and .env.example to document fully-local and mixed cloud/local setups, plus a commented optional ollama Docker Compose service.
  • Add tests for Ollama config shapes and schema validation against mem0.configs.base.MemoryConfig.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
app/config.py Adds ollama_base_url setting (default http://localhost:11434).
app/memory.py Makes provider config provider-aware and wires Ollama base URL + embedding dims into mem0 config.
tests/test_memory.py Adds tests for Ollama-only and mixed provider config shapes + validates against real mem0 schema.
tests/test_config.py Adds test coverage for OLLAMA_BASE_URL default/override.
docs/USER_GUIDE.md Documents Ollama usage, config reference updates, and the embed-dims/Qdrant invariant.
docs/DEVELOPER_GUIDE.md Updates internal notes about _provider_config() behavior for cloud vs Ollama.
docker-compose.yml Adds a commented-out optional ollama service and volume notes.
.env.example Documents Ollama-related env vars and a fully-local example block.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread app/memory.py
Comment thread tests/test_config.py
Addresses PR review: _build_config previously hardcoded the Anthropic key
into the LLM config and the OpenAI key into the embedder config regardless of
the configured provider. With openai now documented as a valid
MEM0_LLM_PROVIDER, an OpenAI LLM would have received the Anthropic key (and no
OpenAI key), silently failing at first request.

- memory.py: add _api_key_for(provider) so each provider gets its own key
  (anthropic->ANTHROPIC_API_KEY, openai->OPENAI_API_KEY, ollama/other->none).
- config.py: _require_provider_keys now requires a provider's key when *either*
  role selects it, so openai-LLM + ollama-embed no longer passes validation
  without OPENAI_API_KEY.
- Tests: openai-LLM key routing, and validation raises for a missing key when
  openai/anthropic is the LLM (or embed) provider.
- Docs: config-table key requirements and the DEVELOPER_GUIDE validator note
  updated to reflect either-role enforcement.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XQXUARh5h5hRFg67toenEX

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Comment thread app/config.py Outdated
Comment thread docs/USER_GUIDE.md
Addresses PR review follow-ups:

- config.py: reword the _require_provider_keys comment — mem0's clients DO
  read keys from os.environ; the reason we inject is that a key loaded from
  .env via pydantic-settings never lands in os.environ. The old wording said
  the opposite and contradicted the note in app/memory.py.
- tests: remove test_missing_anthropic_key_rejected_for_embed_provider. It
  implied Anthropic is a valid embed provider, but Anthropic has no embeddings
  API and the docs list only openai/ollama for embedding — the real key-backed
  cases (anthropic LLM, openai LLM or embed) stay covered. Also drop the same
  misleading os.environ phrasing from the openai-LLM test comment.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XQXUARh5h5hRFg67toenEX
@imonroe
imonroe merged commit a37353a into main Jul 9, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support local Ollama embeddings + LLM provider (cost-free, private, no external API dependency)

3 participants