OlmoEarth Agent turns a natural-language brief into real geospatial work on OlmoEarth Studio. It reasons about the ask, calls tools over a sandboxed Python interpreter, submits and polls predictions, and reports honest results: a provenance manifest per call, mandatory spatial cross-validation on auto-correlated AOIs, and no raw coordinates in chat. It runs on a local Qwen3.6-35B-A3B model served with llama.cpp by default (no hosted LLM required), and can optionally use your own cloud API key via the web UI's model-backend picker.
Prerequisites: uv and an OlmoEarth Studio API key (Studio UI -> profile -> API Keys). The local model path also needs Docker; the cloud-API path needs neither Docker nor a download.
Pick the backbone for the agent's reasoning - both drive the same live UI:
A. Cloud API (Claude / ChatGPT / Gemini) - no Docker, no 17.7 GB download:
make setup # init vendored skills + uv sync --all-extras
make bridge # live web UI on http://localhost:8088 (no local model needed)Open http://localhost:8088, paste your Studio key, then go to Settings -> LLM backend, pick a provider, and paste that provider's API key. Both keys stay client-side and are sent per request, never stored server-side. Send a brief.
B. Local model (fully offline, free) - one command auto-starts the LLM and the UI:
make setup # init vendored skills + uv sync --all-extras
make up # start the 4-bit Qwen3.6 LLM (first run pulls ~17.7 GB), then the live UIOpen http://localhost:8088, paste your Studio key, and send a brief. (make up is just make serve then make bridge; run them separately if you prefer.)
Prefer the terminal? Run a one-shot brief against the local model instead of the web UI:
export OLMOEARTH_API_KEY=... # Studio UI -> profile -> API Keys
make serve && make agent # make agent runs a sample brief; ask your own with Q="..."make help lists every target; make down stops the LLM; make web serves a backend-free demo UI on :8080. LLM_ENDPOINT defaults to http://localhost:8000/v1. No make (e.g. Windows + Git Bash)? ./scripts/quickstart.sh runs setup + serve and prints the exact uv run commands; see docs/serving.md for the cloud-API setup and the raw docker compose invocations.
The LLM reads the brief, plans, and emits a tool call; the harness dispatches it as a function-call tool, feeds the result back, and iterates until it can answer. (An opt-in system:python subprocess is available for light glue between calls.) The operational rules -- trailing-12-month windows, cost guards on fine-tunes, mandatory spatial CV on auto-correlated AOIs, a provenance manifest per call -- are enforced by the harness, not the model.
The capability set ships as 17 skills, grouped by EO-workflow stage.
The 17 skills, by workflow stage (Prep / Configure / Run / Analyze / Integrate / Report)
| # | Skill | What it does | Stage |
|---|---|---|---|
| 1 | olmoearth-data-prep |
Labels (GeoJSON/CSV/Shapefile) -> Studio-importable file (MIME / 10K / multi-metric guards) AND rslearn dataset.json + Lightning YAML, with a 7-criteria audit |
Prep |
| 2 | olmoearth-studio-job-config |
Task description -> Studio wizard answers, 14 presets + cross-field validator | Configure |
| 3 | olmoearth-embeddings |
Embeddings-vs-fine-tune guidance + a runnable notebook, plus the one-call olmoearth_automate (decide + propose a config; optional HuggingFace introspection) |
Configure |
| 4 | olmoearth-predict |
Core run primitive: submit / poll / fetch results; pixel-value / features follow | Run |
| 5 | olmoearth-change-detection |
Two engines: Studio multi-date (>=3) trajectory diff (refuses naive 2-date), and an out-of-process JEPA latent-prediction pixel detector (separate repo olmoearth-jepa-change) |
Run |
| 6 | olmoearth-baseline-compare |
Studio vs. a baseline foundation model, side-by-side on transfer regions | Run |
| 7 | olmoearth-evaluate |
Spatial-block CV + NNDM-LOO over /prediction-results |
Analyze |
| 8 | olmoearth-similarity |
Exact top-K kNN over supplied embeddings (e.g. OlmoEarth Base; FAISS = scale-up follow-up) | Analyze |
| 9 | olmoearth-uncertainty |
Ensemble-disagreement confidence + Meyer-Pebesma Area-of-Applicability (OOD) flag | Analyze |
| 10 | olmoearth-cloud-mask-audit |
CFMask / s2cloudless / Sen2Cor / MAJA ensemble disagreement | Analyze |
| 11 | olmoearth-qgis-bridge |
Tile URLs -> a QGIS .qlr layer + GDAL_WMS descriptor + XYZ URLs + SLD ramp + legend + a local gdal_translate COG recipe (key never embedded) |
Integrate |
| 12 | olmoearth-data-export |
Export Studio projects + predictions to JSON, grouped by project or status | Integrate |
| 13 | olmoearth-provenance |
Manifest wrapper around every API call; emits a replay script | Report |
| 14 | olmoearth-case-narrative |
Stakeholder writeup with live tiles + a freshness gate | Report |
| 15 | olmoearth-litsearch |
arXiv + OpenAlex literature search + DOI/arXiv-id resolution to ground citations | Report |
| 16 | olmoearth-negative-sampler |
Presence-only labels -> trainable set: generates a buffered, spatially-thinned (optionally embedding-dissimilar) negative class so the data-prep audit passes | Prep |
| 17 | olmoearth-rslearn |
Operate rslearn (the data + training engine under OlmoEarth): run the 4-stage pipeline + fit/predict, plus torch-free tools to recommend a full setup from a plain-language goal and validate a config (shapes / label-type / bands) before a multi-hour run | Configure |
See docs/SHOWCASE.md for every skill in action with real outputs -- each driven by the live Qwen3.6 backbone (real reasoning, function calls, and results, not mockups). Per-skill specs are in SKILLS.md; the function catalog, harness dataclasses, and operational rules are in PLAN.md.
The stack, layer by layer
| Layer | What |
|---|---|
| LLM | Default: Qwen3.6-35B-A3B (35B total / 3B active, hybrid Gated-DeltaNet + MoE), served locally as a 4-bit GGUF (unsloth/Qwen3.6-35B-A3B-GGUF UD-IQ4_XS, ~17.7 GB). Optional: bring your own cloud API key, selected in the web UI (model autodetect; key never stored) |
| Serving | llama.cpp (ghcr.io/ggml-org/llama.cpp:server-cuda), OpenAI-compatible API, --jinja for tool calling: see docs/serving.md |
| Harness | A compact catalog of function-call tools + an opt-in system:python subprocess for light glue, with operational constraints enforced (PLAN.md) |
| Skills | 17-skill catalog; vendored #1-#3 + #17 via submodule vendor/olmoearth-skills |
| Studio API | https://olmoearth.allenai.org/api/v1, Bearer OLMOEARTH_API_KEY (docs) |
- PLAN.md: the function catalog, harness dataclasses, and operational rules
- SKILLS.md: the full 17-skill catalog (Prep / Configure / Run / Analyze / Integrate / Report)
- docs/SHOWCASE.md: every skill run live, with real outputs
- docs/CANON.md: the canonical facts the repo holds itself to
- CONTRIBUTING.md | CHANGELOG.md | AGENTS.md
OlmoEarth Artifact License (Ai2) - free use with restrictions: no military/defense/surveillance or extractive uses; cite Ai2 and propagate the terms downstream. Matches allenai/olmoearth_pretrain.