From 5694c89873c14716cfc50e794285ee253c4cf2a7 Mon Sep 17 00:00:00 2001 From: Sina Date: Tue, 26 May 2026 09:22:17 -0400 Subject: [PATCH 1/2] docs: Mimir on Juju docs --- docs/how-to/deploy-and-manage/install.md | 6 + .../configure-object-storage-for-mimir.md | 120 +++++++++++ docs/how-to/integrate/index.md | 4 +- .../how-to/integrate/send-metrics-to-mimir.md | 154 ++++++++++++++ docs/tutorial/index.md | 2 +- docs/tutorial/mimir-on-juju.md | 192 ++++++++++++++++++ 6 files changed, 476 insertions(+), 2 deletions(-) create mode 100644 docs/how-to/integrate/configure-object-storage-for-mimir.md create mode 100644 docs/how-to/integrate/send-metrics-to-mimir.md create mode 100644 docs/tutorial/mimir-on-juju.md diff --git a/docs/how-to/deploy-and-manage/install.md b/docs/how-to/deploy-and-manage/install.md index a83ab5cd..6007a6ca 100644 --- a/docs/how-to/deploy-and-manage/install.md +++ b/docs/how-to/deploy-and-manage/install.md @@ -8,6 +8,12 @@ myst: This guide walks you through the prerequisites and deployment planning needed to install the Canonical Observability Stack (COS). Use it to prepare your environment, create a Terraform plan, and deploy supporting components in the recommended order. +```{note} +If you are coming from upstream Grafana Mimir docs and only want to deploy +standalone Mimir on Juju, start with +[Get started with standalone Mimir on Juju](/tutorial/mimir-on-juju). +``` + ## Preparation Before deploying COS or COS Lite, work through the items below. diff --git a/docs/how-to/integrate/configure-object-storage-for-mimir.md b/docs/how-to/integrate/configure-object-storage-for-mimir.md new file mode 100644 index 00000000..2c956d56 --- /dev/null +++ b/docs/how-to/integrate/configure-object-storage-for-mimir.md @@ -0,0 +1,120 @@ +--- +myst: + html_meta: + description: "Configure S3-compatible object storage for standalone Mimir on Juju, including s3-integrator setup, production considerations, and Minio-based testing links." +--- + +# How to configure object storage for Mimir + +Mimir on Juju needs an S3-compatible object store for durable storage. +The Juju deployment does **not** create that object store for you; it expects +you to provide one and then connect it through `s3-integrator`. + +Use this guide when you already have an object store and want to wire it into a +standalone Mimir deployment. + +## Before you begin + +Have the following details ready: + +- S3 endpoint +- bucket name +- access key +- secret key +- optional region +- optional custom CA chain for HTTPS + +If you still need a local test object store, see +[How to deploy Minio and S3 Integrator](deploy-s3-integrator-and-minio). + +```{warning} +The Minio charm is useful for testing, but not as a production storage +strategy. For production planning, also review +[Storage best practices](/reference/storage). +``` + +## 1. Deploy the S3 integrator + +```bash +juju deploy s3-integrator mimir-s3 --channel latest/stable --trust +``` + +The application will remain blocked until you give it credentials and a target +bucket. + +## 2. Create and grant the S3 credentials secret + +Create a Juju secret holding the access key and secret key, then grant that +secret to the `mimir-s3` application: + +```bash +juju add-secret mimir-s3-credentials \ + access-key= \ + secret-key= + +juju grant-secret secret: mimir-s3 +``` + +## 3. Configure the object-store location + +Set the endpoint and bucket: + +```bash +juju config mimir-s3 \ + credentials=secret: \ + endpoint= \ + bucket= +``` + +Depending on your object-store implementation, you may also need additional +options such as: + +- `region=` +- `path=` +- `s3-uri-style=path` + +If the endpoint uses a private CA, provide it as base64: + +```bash +juju config mimir-s3 tls-ca-chain="$(base64 -w0 ca-chain.pem)" +``` + +## 4. Integrate Mimir with the S3 integrator + +```bash +juju integrate mimir:s3 mimir-s3:s3-credentials +``` + +If you are still building the deployment, integrate storage **before** joining +Mimir workers to the coordinator. That keeps the storage configuration in place +before the workers read their cluster settings. + +## 5. Verify the relation + +```bash +juju status --relations +``` + +You should see an `s3` integration between `mimir` and `mimir-s3`. + +## Recommended bucket layout + +Keep Mimir in its own bucket unless you have a clear reason to share one. + +A simple starting point is: + +| Purpose | Recommendation | +|---|---| +| Bucket | Dedicated bucket per Mimir deployment | +| Credentials | Dedicated credentials per deployment | +| TLS | Prefer HTTPS in shared or production environments | +| Testing | Use Minio only for local or disposable environments | + +## Production notes + +- Plan object-store capacity around your retention and ingestion rates. +- Avoid local-only or node-bound storage for production-grade deployments. +- Keep the S3 endpoint stable before you start wiring producers to + `receive-remote-write`. + +For broader sizing and storage guidance, see [Storage best practices](/reference/storage). diff --git a/docs/how-to/integrate/index.md b/docs/how-to/integrate/index.md index f351de31..3db80b05 100644 --- a/docs/how-to/integrate/index.md +++ b/docs/how-to/integrate/index.md @@ -18,6 +18,7 @@ Monitor SSL certificates with blackbox exporter Use the Loki HTTP API Use Catalogue +Send metrics to Mimir ``` ## Extend the pipeline @@ -41,10 +42,11 @@ Sync alert rules from a git repo ## Test & validate integrations -Verify your object-storage integration with a local Minio instance. +Validate local object-storage integrations or wire existing storage into Mimir. ```{toctree} :maxdepth: 1 Testing with Minio +Configure object storage for Mimir ``` diff --git a/docs/how-to/integrate/send-metrics-to-mimir.md b/docs/how-to/integrate/send-metrics-to-mimir.md new file mode 100644 index 00000000..c3c541f7 --- /dev/null +++ b/docs/how-to/integrate/send-metrics-to-mimir.md @@ -0,0 +1,154 @@ +--- +myst: + html_meta: + description: "Send metrics from uncharmed workloads to standalone Mimir on Juju by using the OpenTelemetry Collector snap as a scraper and remote_write client." +--- + +# How to send metrics to Mimir + +Use this guide when your workload is **not** managed by Juju and you want to +ship its metrics into standalone Mimir. + +The recommended path is to run the OpenTelemetry Collector snap near the +uncharmed workload, scrape its `/metrics` endpoint, and forward those metrics +to Mimir over Prometheus `remote_write`. + +For the standalone Mimir deployment itself, see +[Get started with standalone Mimir on Juju](/tutorial/mimir-on-juju). + +## Prerequisites + +- A working standalone Mimir deployment with ingress enabled. +- An uncharmed workload that exposes Prometheus-format metrics. +- A machine where you can install the OpenTelemetry Collector snap and that can: + - reach the workload metrics endpoint + - reach the Mimir ingress URL + +## 1. Get the Mimir URL + +In the Juju model where Mimir is deployed, run: + +```bash +juju run traefik/0 show-proxied-endpoints --format=yaml \ + | yq '."traefik/0".results."proxied-endpoints"' \ + | jq +``` + +Look for the `mimir` entry: + +```json +{ + "mimir": { + "url": "http://10.43.8.34:80/mimir" + } +} +``` + +You will use: + +- `/api/v1/push` for ingestion +- `/prometheus/api/v1/query` for verification + +## 2. Install the OpenTelemetry Collector snap + +On the machine that can reach the uncharmed workload: + +```bash +sudo snap install opentelemetry-collector +``` + +We recommend placing the collector as close as possible to the workload to +reduce network hops and avoid losing telemetry during transient failures. + +## 3. Create the collector configuration + +Write a collector config file under `/etc/otelcol/config.d/`. + +This example scrapes a workload that exposes metrics on `http://my-workload:9000/metrics` +and forwards them to Mimir: + +```yaml +receivers: + prometheus: + config: + scrape_configs: + - job_name: "my-workload" + scrape_interval: 15s + static_configs: + - targets: ["my-workload:9000"] + +exporters: + prometheusremotewrite: + endpoint: "http://10.43.8.34:80/mimir/api/v1/push" + headers: + X-Scope-OrgID: anonymous + +service: + pipelines: + metrics: + receivers: [prometheus] + exporters: [prometheusremotewrite] +``` + +Save it as: + +```bash +sudo mkdir -p /etc/otelcol/config.d +sudo editor /etc/otelcol/config.d/otelcol_mimir.yaml +``` + +Replace: + +- `my-workload:9000` with the address of your workload's metrics endpoint +- `http://10.43.8.34:80/mimir/api/v1/push` with your actual Mimir ingress URL + +## 4. Restart the collector + +```bash +sudo snap restart opentelemetry-collector +``` + +If you want to confirm the collector started cleanly: + +```bash +sudo snap logs opentelemetry-collector +``` + +## 5. Verify that metrics reached Mimir + +Query Mimir's Prometheus-compatible API: + +```bash +MIMIR_URL=http://10.43.8.34:80/mimir + +curl -sG "$MIMIR_URL/prometheus/api/v1/query" \ + --data-urlencode 'query=up{job="my-workload"}' \ + | jq +``` + +If the pipeline is working, the query returns a successful result for the +`my-workload` job. + +## HTTPS and private CAs + +If either the workload metrics endpoint or the Mimir ingress URL uses TLS signed +by a private CA, install that CA into the machine trust store used by the snap. + +See the CA-handling guidance in +[How to integrate COS Lite with uncharmed applications](integrating-cos-lite-with-uncharmed-applications). + +## Adapting the scrape config + +The example above uses a single static target, but the same pattern works for: + +- multiple targets in one job +- multiple scrape jobs +- custom `metrics_path` values +- authenticated or HTTPS endpoints + +The important part is that the metrics pipeline ends with the +`prometheusremotewrite` exporter pointing at: + +```text +/api/v1/push +``` diff --git a/docs/tutorial/index.md b/docs/tutorial/index.md index 78af36c7..9fe39a95 100644 --- a/docs/tutorial/index.md +++ b/docs/tutorial/index.md @@ -25,6 +25,7 @@ Production deployment configuration may differ from the setup showcased here. ```{toctree} :maxdepth: 1 +Standalone Mimir on Juju COS on Canonical K8s ``` @@ -36,4 +37,3 @@ COS on Canonical K8s COS Lite on Canonical K8s COS Lite on MicroK8s ``` - diff --git a/docs/tutorial/mimir-on-juju.md b/docs/tutorial/mimir-on-juju.md new file mode 100644 index 00000000..7421aa5f --- /dev/null +++ b/docs/tutorial/mimir-on-juju.md @@ -0,0 +1,192 @@ +--- +myst: + html_meta: + description: "Deploy standalone Grafana Mimir on Juju with a small worker topology, S3-backed storage, and a minimal remote_write validation path." +--- + +# Get started with standalone Mimir on Juju + +This tutorial is for readers who already know upstream Grafana Mimir and want +the Juju-native path to a working deployment. + +You will deploy a small standalone Mimir topology on Kubernetes, back it with +S3-compatible object storage, expose it through Traefik, and confirm that it +can ingest metrics over `prometheus_remote_write`. + +```{important} +This guide uses the Juju CLI because it is the clearest way to learn how the +deployment fits together. For repeatable environments and production-oriented +rollouts, prefer the +[Mimir Terraform module](https://github.com/canonical/mimir-operators/tree/main/terraform). +``` + +## Prerequisites + +- Juju 3.6 or later, with a Kubernetes cloud added and a controller already + bootstrapped. +- A Kubernetes cluster ready to host the Mimir model. +- An S3-compatible object store that already exists, including: + - endpoint + - bucket name + - access key + - secret key +- A routable ingress address for Traefik if you want to query Mimir from + outside the cluster. + +If you still need to prepare storage, see +[How to configure object storage for Mimir](../how-to/integrate/configure-object-storage-for-mimir). + +## 1. Create a model + +```bash +juju add-model mimir +juju switch mimir +``` + +## 2. Deploy Mimir and its supporting charms + +Deploy the coordinator, one worker application, Traefik, and an S3 integrator. + +This tutorial uses a compact worker layout for ease of demonstration. The +single `mimir-worker` application runs with `role-all=true`, which keeps the +deployment small while still exercising the coordinator, storage, ingress, and +ingestion flows. + +For higher availability and more production-like scaling, split the worker tier +into dedicated `mimir-write`, `mimir-read`, and `mimir-backend` applications +instead of using this monolithic worker setup. You can then scale each role +independently according to ingestion, query, and storage pressure. + +The tutorial worker also uses `role-query-frontend=true` so queries go through +the same frontend path you will use later. + +```bash +juju deploy mimir-coordinator-k8s mimir --trust + +juju deploy mimir-worker-k8s mimir-worker \ + --trust \ + --config role-all=true \ + --config role-query-frontend=true + +juju deploy s3-integrator mimir-s3 --channel latest/stable --trust +juju deploy traefik-k8s traefik --trust +``` + +## 3. Point the S3 integrator at your object storage + +Replace the placeholder values with your existing S3-compatible storage +details: + +```bash +juju add-secret mimir-s3-credentials \ + access-key= \ + secret-key= + +juju grant-secret mimir-s3-credentials mimir-s3 + +juju config mimir-s3 \ + credentials=mimir-s3-credentials \ + endpoint= \ + bucket= +``` + +If your object store needs additional settings such as a custom CA chain, +region, or path-style addressing, see +[How to configure object storage for Mimir](../how-to/integrate/configure-object-storage-for-mimir). + +## 4. Integrate the deployment + +Connect storage first, then join the worker and ingress relations: + +```bash +juju integrate mimir:s3 mimir-s3:s3-credentials +juju integrate mimir:mimir-cluster mimir-worker:mimir-cluster +juju integrate traefik:ingress mimir:ingress +``` + +Watch the model until all applications settle: + +```bash +juju status --relations --watch=5s +``` + +At this point Mimir should expose at least: + +- `receive-remote-write` for metrics ingestion +- `ingress` for external access +- `self-metrics-endpoint` for scrape-based validation flows + +## 5. Validate metric ingestion with OpenTelemetry Collector + +For a minimal smoke test, deploy: + +- `avalanche-k8s` as a small metrics generator +- `opentelemetry-collector-k8s` as the sender that scrapes Avalanche and + forwards the metrics to Mimir over `prometheus_remote_write` + +```bash +juju deploy opentelemetry-collector-k8s otelcol --trust +juju deploy avalanche-k8s avalanche --trust + +juju integrate avalanche otelcol:metrics-endpoint +juju integrate otelcol:send-remote-write mimir:receive-remote-write +``` + +Wait for the applications to become active: + +```bash +juju status --relations --watch=5s +``` + +## 6. Query Mimir + +Use the Traefik action to get the Mimir URL: + +```bash +juju run traefik/0 show-proxied-endpoints --format=yaml \ + | yq '."traefik/0".results."proxied-endpoints"' \ + | jq +``` + +The output includes a `mimir` entry similar to: + +```json +{ + "mimir": { + "url": "http://10.43.8.34:80/mimir" + } +} +``` + +Save that URL and query Mimir's Prometheus-compatible API: + +```bash +MIMIR_URL=http://10.43.8.34:80/mimir + +curl -sG "$MIMIR_URL/prometheus/api/v1/query" \ + --data-urlencode 'query=count({__name__=~"avalanche_metric_.+"})' \ + | jq +``` + +You should see a successful query response with a value greater than `0`. + +## What you deployed + +This tutorial uses a small Juju-native Mimir deployment: + +- `mimir-coordinator-k8s` provides the public API surface +- `mimir-worker-k8s` runs the actual Mimir workload roles +- `s3-integrator` supplies object-store connection details to Mimir +- `traefik-k8s` gives you a stable URL for querying and ingestion + +That is enough to run Mimir standalone. It also fits naturally into the wider +COS architecture if you later add Grafana, Loki, Alertmanager, or more +specialized telemetry pipelines. + +## Next steps + +- For storage details and production-oriented caveats, see + [How to configure object storage for Mimir](../how-to/integrate/configure-object-storage-for-mimir). +- For more ways to send metrics into Mimir, including cross-model relations and + direct `remote_write` clients, see + [How to send metrics to Mimir](../how-to/integrate/send-metrics-to-mimir). From 7de1c938db8b4d9755a5936efa6b52357b17566a Mon Sep 17 00:00:00 2001 From: Luca Bello Date: Tue, 7 Jul 2026 16:33:28 +0200 Subject: [PATCH 2/2] docs: address review feedback on Mimir on Juju guides Restructure the Mimir on Juju documentation set based on review: - Convert the Mimir on Juju guide from a tutorial to a how-to, moved under how-to/deploy-and-manage, wired into that index, and removed from the tutorial index. Redirect kept for the old URL. - Drop the "standalone Mimir" framing across all three docs. - Add a Background section covering Juju terminology and clarify which charms are Mimir infrastructure vs. supporting infrastructure. - Move planning content to the top of the object-storage how-to and rename it to "connect object storage" to match what it does. - Rewrite the send-metrics intro and add context on why the guide uses the OpenTelemetry Collector snap instead of a charm. - Replace the top-of-page note in the install guide with a "Deploy individual components" section at the bottom. --- .../deploy-and-manage/deploy-mimir-on-juju.md | 229 ++++++++++++++++++ docs/how-to/deploy-and-manage/index.md | 1 + docs/how-to/deploy-and-manage/install.md | 13 +- .../configure-object-storage-for-mimir.md | 91 +++---- docs/how-to/integrate/index.md | 4 +- .../how-to/integrate/send-metrics-to-mimir.md | 69 ++++-- docs/redirects.txt | 1 + docs/tutorial/index.md | 1 - docs/tutorial/mimir-on-juju.md | 192 --------------- 9 files changed, 337 insertions(+), 264 deletions(-) create mode 100644 docs/how-to/deploy-and-manage/deploy-mimir-on-juju.md delete mode 100644 docs/tutorial/mimir-on-juju.md diff --git a/docs/how-to/deploy-and-manage/deploy-mimir-on-juju.md b/docs/how-to/deploy-and-manage/deploy-mimir-on-juju.md new file mode 100644 index 00000000..c920ff79 --- /dev/null +++ b/docs/how-to/deploy-and-manage/deploy-mimir-on-juju.md @@ -0,0 +1,229 @@ +--- +myst: + html_meta: + description: "Deploy Grafana Mimir on Juju with a small worker topology, S3-backed storage, and a minimal remote_write validation path, without the full COS bundle." +--- + +# How to deploy Mimir on Juju + +This guide deploys Grafana Mimir on Kubernetes using +[Juju](https://documentation.ubuntu.com/juju/3.6/), Canonical's operator +lifecycle manager, without pulling in the full Canonical Observability Stack +(COS) bundle. It targets readers who already know upstream Mimir and want the +Juju-native path to a working, Mimir-focused deployment. + +If you are new to Juju, skim the [Juju documentation](https://documentation.ubuntu.com/juju/3.6/) +first — especially [Juju: get started](https://documentation.ubuntu.com/juju/3.6/tutorial/) +and the [Juju reference](https://documentation.ubuntu.com/juju/3.6/reference/) +for the meaning of terms like *model*, *application*, *unit*, and *relation*. +The runtime you get from this guide is upstream Mimir; the *charms* just handle +configuration, storage wiring, ingress, and inter-component relations. + +For a full COS deployment (Grafana, Alertmanager, Loki, etc.) instead of a +Mimir-focused one, see +[Getting started with COS on Canonical K8s](/tutorial/cos-canonical-k8s-sandbox). + +```{important} +This guide uses the Juju CLI because it is the clearest way to see how the +deployment fits together. For repeatable environments and production-oriented +rollouts, prefer the +[Mimir Terraform module](https://github.com/canonical/mimir-operators/tree/main/terraform). +``` + +## What this guide deploys + +Mimir infrastructure (required for Mimir to run): + +- `mimir-coordinator-k8s` — the public API and coordinator for the Mimir + cluster. +- `mimir-worker-k8s` — a single worker application configured to run all + Mimir roles (`role-all=true`). In production, split this into dedicated + `mimir-write`, `mimir-read`, and `mimir-backend` applications and scale each + role independently. +- `s3-integrator` (deployed as `mimir-s3`) — supplies Mimir with an S3 + endpoint and credentials. Mimir requires S3-compatible object storage; the + charm does not provide the object store itself, only the connection + details. + +Supporting infrastructure (not part of Mimir itself, but used here for a +usable end-to-end flow): + +- `traefik-k8s` — gives Mimir a stable URL for ingestion and querying from + outside the cluster. +- `opentelemetry-collector-k8s` (`otelcol`) — used only in the validation + step to scrape a test workload and push metrics to Mimir over + `prometheus_remote_write`. +- `avalanche-k8s` — a synthetic metrics generator used only in the + validation step. + +The worker also uses `role-query-frontend=true` so queries you run later go +through the same frontend path as a real deployment. + +## Prerequisites + +- Juju 3.6 or later, with a Kubernetes cloud added and a controller + bootstrapped. See + [Juju: install Juju](https://documentation.ubuntu.com/juju/3.6/howto/manage-juju/#install-juju), + [Juju: add a Kubernetes cloud](https://documentation.ubuntu.com/juju/3.6/howto/manage-clouds/#add-a-kubernetes-cloud), + and [Juju: bootstrap a controller](https://documentation.ubuntu.com/juju/3.6/howto/manage-controllers/). +- A Kubernetes cluster ready to host the Mimir model. For a local option, + see [Canonical Kubernetes: get started](https://documentation.ubuntu.com/canonical-kubernetes/latest/snap/tutorial/getting-started/). +- An S3-compatible object store that already exists, with an endpoint, + bucket, access key, and secret key. If you do not have one, see + [How to deploy Minio and S3 Integrator](../integrate/deploy-s3-integrator-and-minio) + for a disposable test store. For the connection steps in more detail, see + [How to connect object storage to Mimir on Juju](../integrate/configure-object-storage-for-mimir). +- A routable ingress address for Traefik if you want to query Mimir from + outside the cluster. See the [networking best practices](/reference/networking). + +## 1. Create a model + +A Juju [*model*](https://documentation.ubuntu.com/juju/3.6/reference/model/) +is a workspace on the controller that holds a set of related applications. +Create one dedicated to Mimir: + +```bash +juju add-model mimir +juju switch mimir +``` + +## 2. Deploy Mimir and its supporting charms + +Deploy the Mimir coordinator, one worker application, an S3 integrator, and +Traefik: + +```bash +juju deploy mimir-coordinator-k8s mimir --trust + +juju deploy mimir-worker-k8s mimir-worker \ + --trust \ + --config role-all=true \ + --config role-query-frontend=true + +juju deploy s3-integrator mimir-s3 --channel latest/stable --trust +juju deploy traefik-k8s traefik --trust +``` + +`--trust` grants the charm access to the Kubernetes API on the host cluster, +which the Mimir, S3 integrator, and Traefik charms need to create the +resources they manage. + +## 3. Point the S3 integrator at your object storage + +The `mimir-s3` application starts in `blocked` status until it has both +credentials and a target bucket. Create a Juju secret with the S3 +credentials, grant that secret to `mimir-s3`, then configure the endpoint +and bucket. + +Replace the placeholders with your existing S3-compatible storage details: + +```bash +juju add-secret mimir-s3-credentials \ + access-key= \ + secret-key= + +juju grant-secret mimir-s3-credentials mimir-s3 + +juju config mimir-s3 \ + credentials=mimir-s3-credentials \ + endpoint= \ + bucket= +``` + +If your object store needs additional settings such as a custom CA chain, +region, or path-style addressing, see +[How to connect object storage to Mimir on Juju](../integrate/configure-object-storage-for-mimir). + +## 4. Integrate the deployment + +Connect storage first, then join the worker and ingress relations: + +```bash +juju integrate mimir:s3 mimir-s3:s3-credentials +juju integrate mimir:mimir-cluster mimir-worker:mimir-cluster +juju integrate traefik:ingress mimir:ingress +``` + +Watch the model until every application reaches `active/idle`. That is the +Juju status meaning the charm has finished reconciling and is running as +configured; other transient statuses (`waiting`, `maintenance`, `blocked`) +indicate the deployment is still converging or missing something: + +```bash +juju status --relations --watch=5s +``` + +At this point Mimir should expose at least: + +- `receive-remote-write` for metrics ingestion +- `ingress` for external access +- `self-metrics-endpoint` for scrape-based validation flows + +## 5. Validate metric ingestion with OpenTelemetry Collector + +For a minimal smoke test, deploy: + +- `avalanche-k8s` — a small synthetic metrics generator. +- `opentelemetry-collector-k8s` — a scraper and forwarder charm that + collects Avalanche's metrics and pushes them to Mimir over + `prometheus_remote_write`. + +Neither of these is required for Mimir itself. They only exist here to prove +the ingestion path works end-to-end: + +```bash +juju deploy opentelemetry-collector-k8s otelcol --trust +juju deploy avalanche-k8s avalanche --trust + +juju integrate avalanche otelcol:metrics-endpoint +juju integrate otelcol:send-remote-write mimir:receive-remote-write +``` + +Wait for the applications to reach `active/idle`: + +```bash +juju status --relations --watch=5s +``` + +## 6. Query Mimir + +Use the Traefik action to get the Mimir URL: + +```bash +juju run traefik/0 show-proxied-endpoints --format=yaml \ + | yq '."traefik/0".results."proxied-endpoints"' \ + | jq +``` + +The output includes a `mimir` entry similar to: + +```json +{ + "mimir": { + "url": "http://10.43.8.34:80/mimir" + } +} +``` + +Save that URL and query Mimir's Prometheus-compatible API: + +```bash +MIMIR_URL=http://10.43.8.34:80/mimir + +curl -sG "$MIMIR_URL/prometheus/api/v1/query" \ + --data-urlencode 'query=count({__name__=~"avalanche_metric_.+"})' \ + | jq +``` + +You should see a successful query response with a value greater than `0`. + +## Next steps + +- For storage details and production-oriented caveats, see + [How to connect object storage to Mimir on Juju](../integrate/configure-object-storage-for-mimir). +- For more ways to send metrics into Mimir, including cross-model relations + and direct `remote_write` clients, see + [How to send metrics to Mimir on Juju](../integrate/send-metrics-to-mimir). +- To grow this into a full Canonical Observability Stack with Grafana, Loki, + Alertmanager, and correlated telemetry, see + [Getting started with COS on Canonical K8s](/tutorial/cos-canonical-k8s-sandbox). diff --git a/docs/how-to/deploy-and-manage/index.md b/docs/how-to/deploy-and-manage/index.md index 04d7166c..9e961558 100644 --- a/docs/how-to/deploy-and-manage/index.md +++ b/docs/how-to/deploy-and-manage/index.md @@ -21,6 +21,7 @@ Install Configure strict reproducibility Configure the Juju model Configure the Grafana database +Deploy Mimir on Juju ``` ## Secure access diff --git a/docs/how-to/deploy-and-manage/install.md b/docs/how-to/deploy-and-manage/install.md index 6007a6ca..3d970715 100644 --- a/docs/how-to/deploy-and-manage/install.md +++ b/docs/how-to/deploy-and-manage/install.md @@ -8,12 +8,6 @@ myst: This guide walks you through the prerequisites and deployment planning needed to install the Canonical Observability Stack (COS). Use it to prepare your environment, create a Terraform plan, and deploy supporting components in the recommended order. -```{note} -If you are coming from upstream Grafana Mimir docs and only want to deploy -standalone Mimir on Juju, start with -[Get started with standalone Mimir on Juju](/tutorial/mimir-on-juju). -``` - ## Preparation Before deploying COS or COS Lite, work through the items below. @@ -103,3 +97,10 @@ where `.n` in `tf-cos-3.0.n` is the latest available patch version in the [COS t ## Deploy COS Alerter COS Alerter is a watchdog service for COS. Deploy it on dedicated infrastructure that is separate from your COS or COS Lite deployment. For more information, including deployment details, see the [COS Alerter repository](https://github.com/canonical/cos-alerter). + +## Deploy individual components + +If you are not deploying the full Canonical Observability Stack and only need a +specific COS component on Juju, see the component-focused guides: + +- [How to deploy Mimir on Juju](deploy-mimir-on-juju) diff --git a/docs/how-to/integrate/configure-object-storage-for-mimir.md b/docs/how-to/integrate/configure-object-storage-for-mimir.md index 2c956d56..99a7e5f6 100644 --- a/docs/how-to/integrate/configure-object-storage-for-mimir.md +++ b/docs/how-to/integrate/configure-object-storage-for-mimir.md @@ -1,35 +1,61 @@ --- myst: html_meta: - description: "Configure S3-compatible object storage for standalone Mimir on Juju, including s3-integrator setup, production considerations, and Minio-based testing links." + description: "Connect S3-compatible object storage to a Mimir on Juju deployment through s3-integrator, with production planning notes and Minio-based testing links." --- -# How to configure object storage for Mimir +# How to connect object storage to Mimir on Juju -Mimir on Juju needs an S3-compatible object store for durable storage. -The Juju deployment does **not** create that object store for you; it expects -you to provide one and then connect it through `s3-integrator`. +Mimir on Juju needs an S3-compatible object store for durable storage. The +Juju deployment does **not** create that object store for you; it expects you +to bring an existing one and connect it through the `s3-integrator` charm. -Use this guide when you already have an object store and want to wire it into a -standalone Mimir deployment. +Use this guide to connect an existing object store to a Mimir on Juju +deployment. ## Before you begin -Have the following details ready: +You should already have: -- S3 endpoint -- bucket name -- access key -- secret key -- optional region -- optional custom CA chain for HTTPS +- A Mimir on Juju deployment (at minimum the `mimir-coordinator-k8s` + application). If you do not have one yet, follow + [How to deploy Mimir on Juju](/how-to/deploy-and-manage/deploy-mimir-on-juju) first. +- Access to an S3-compatible object store, with: + - endpoint + - bucket name + - access key + - secret key + - optional region + - optional custom CA chain for HTTPS If you still need a local test object store, see [How to deploy Minio and S3 Integrator](deploy-s3-integrator-and-minio). +## Plan your object storage + +Do this **before** wiring anything into Mimir; changing the answer to any of +these later is disruptive. + +- **Use a dedicated bucket.** Keep Mimir in its own bucket unless you have a + clear reason to share one. +- **Use dedicated credentials** for that bucket, scoped so they can only + access what Mimir needs. +- **Prefer HTTPS** for the S3 endpoint in shared or production environments, + and know in advance whether the endpoint uses a private CA (you will need + to supply the CA chain during configuration). +- **Do not use Minio in production.** The Minio charm is only for local or + disposable test environments; production planning is covered in + [Storage best practices](/reference/storage). +- **Plan capacity** around your retention and ingestion rates, and avoid + local-only or node-bound storage for production deployments. Sizing details + are in [Storage best practices](/reference/storage). +- **Keep the S3 endpoint stable** before you start wiring producers to + `receive-remote-write`; changing it later forces a reconfiguration of + Mimir workers. + ```{warning} The Minio charm is useful for testing, but not as a production storage -strategy. For production planning, also review +strategy. For production planning, review [Storage best practices](/reference/storage). ``` @@ -39,8 +65,8 @@ strategy. For production planning, also review juju deploy s3-integrator mimir-s3 --channel latest/stable --trust ``` -The application will remain blocked until you give it credentials and a target -bucket. +The application will remain blocked until you give it credentials and a +target bucket. ## 2. Create and grant the S3 credentials secret @@ -85,9 +111,9 @@ juju config mimir-s3 tls-ca-chain="$(base64 -w0 ca-chain.pem)" juju integrate mimir:s3 mimir-s3:s3-credentials ``` -If you are still building the deployment, integrate storage **before** joining -Mimir workers to the coordinator. That keeps the storage configuration in place -before the workers read their cluster settings. +If you are still building the deployment, integrate storage **before** +joining Mimir workers to the coordinator. That keeps the storage +configuration in place before the workers read their cluster settings. ## 5. Verify the relation @@ -95,26 +121,5 @@ before the workers read their cluster settings. juju status --relations ``` -You should see an `s3` integration between `mimir` and `mimir-s3`. - -## Recommended bucket layout - -Keep Mimir in its own bucket unless you have a clear reason to share one. - -A simple starting point is: - -| Purpose | Recommendation | -|---|---| -| Bucket | Dedicated bucket per Mimir deployment | -| Credentials | Dedicated credentials per deployment | -| TLS | Prefer HTTPS in shared or production environments | -| Testing | Use Minio only for local or disposable environments | - -## Production notes - -- Plan object-store capacity around your retention and ingestion rates. -- Avoid local-only or node-bound storage for production-grade deployments. -- Keep the S3 endpoint stable before you start wiring producers to - `receive-remote-write`. - -For broader sizing and storage guidance, see [Storage best practices](/reference/storage). +You should see an `s3` integration between `mimir` and `mimir-s3`, and both +applications should reach `active/idle`. diff --git a/docs/how-to/integrate/index.md b/docs/how-to/integrate/index.md index 3db80b05..9442bca4 100644 --- a/docs/how-to/integrate/index.md +++ b/docs/how-to/integrate/index.md @@ -18,7 +18,7 @@ Monitor SSL certificates with blackbox exporter Use the Loki HTTP API Use Catalogue -Send metrics to Mimir +Send metrics to Mimir on Juju ``` ## Extend the pipeline @@ -48,5 +48,5 @@ Validate local object-storage integrations or wire existing storage into Mimir. :maxdepth: 1 Testing with Minio -Configure object storage for Mimir +Connect object storage to Mimir on Juju ``` diff --git a/docs/how-to/integrate/send-metrics-to-mimir.md b/docs/how-to/integrate/send-metrics-to-mimir.md index c3c541f7..d6bcec25 100644 --- a/docs/how-to/integrate/send-metrics-to-mimir.md +++ b/docs/how-to/integrate/send-metrics-to-mimir.md @@ -1,26 +1,49 @@ --- myst: html_meta: - description: "Send metrics from uncharmed workloads to standalone Mimir on Juju by using the OpenTelemetry Collector snap as a scraper and remote_write client." + description: "Send metrics from applications outside Juju into a Mimir on Juju deployment by using the OpenTelemetry Collector snap as a scraper and remote_write client." --- -# How to send metrics to Mimir +# How to send metrics to Mimir on Juju -Use this guide when your workload is **not** managed by Juju and you want to -ship its metrics into standalone Mimir. +Use this guide to send metrics from an application that is **not** managed by +Juju into a Mimir on Juju deployment. -The recommended path is to run the OpenTelemetry Collector snap near the -uncharmed workload, scrape its `/metrics` endpoint, and forward those metrics -to Mimir over Prometheus `remote_write`. +## What this guide sets up -For the standalone Mimir deployment itself, see -[Get started with standalone Mimir on Juju](/tutorial/mimir-on-juju). +Mimir on Juju accepts metrics over Prometheus `remote_write` through its +`receive-remote-write` relation. When the sender is another charm, that +relation does everything for you. When the sender is an uncharmed workload +(for example, a plain systemd service on a VM), there is no relation to +attach to, so you need something outside Juju that can: + +1. scrape or receive metrics from the workload, and +2. push them into Mimir's `remote_write` endpoint through the Mimir + ingress URL. + +We use the [OpenTelemetry Collector +snap](https://snapcraft.io/opentelemetry-collector) for this. The **snap** +(not the charm) is a standalone binary you install directly on the machine +that runs, or can reach, the uncharmed workload. It is configured through a +YAML file placed in `/etc/otelcol/config.d/`. We recommend running it as +close as possible to the workload to minimize network hops and reduce the +chance of losing telemetry during transient failures. + +The end result is: `uncharmed workload` -> `otelcol snap` -(remote_write over +Traefik ingress)-> `Mimir on Juju`. + +For a general primer on this pattern with the full COS Lite stack, see +[How to integrate COS Lite with uncharmed applications](integrating-cos-lite-with-uncharmed-applications). +For the Mimir deployment itself, see +[How to deploy Mimir on Juju](/how-to/deploy-and-manage/deploy-mimir-on-juju). ## Prerequisites -- A working standalone Mimir deployment with ingress enabled. +- A working Mimir on Juju deployment with ingress enabled (see + [How to deploy Mimir on Juju](/how-to/deploy-and-manage/deploy-mimir-on-juju)). - An uncharmed workload that exposes Prometheus-format metrics. -- A machine where you can install the OpenTelemetry Collector snap and that can: +- A machine where you can install the OpenTelemetry Collector snap and that + can: - reach the workload metrics endpoint - reach the Mimir ingress URL @@ -57,15 +80,19 @@ On the machine that can reach the uncharmed workload: sudo snap install opentelemetry-collector ``` -We recommend placing the collector as close as possible to the workload to -reduce network hops and avoid losing telemetry during transient failures. - ## 3. Create the collector configuration -Write a collector config file under `/etc/otelcol/config.d/`. +Write a collector config file under `/etc/otelcol/config.d/`. The snap loads +every YAML file it finds in that directory and merges them, so you can drop +in additional files later for more workloads without editing the existing +one. -This example scrapes a workload that exposes metrics on `http://my-workload:9000/metrics` -and forwards them to Mimir: +The pipeline below scrapes a workload that exposes metrics on +`http://my-workload:9000/metrics` and forwards them to Mimir over +`remote_write`. The `prometheus` receiver acts as the scraper, and the +`prometheusremotewrite` exporter pushes into Mimir's ingestion URL. The +`X-Scope-OrgID` header is required by Mimir's multi-tenant API; use +`anonymous` for a single-tenant deployment. ```yaml receivers: @@ -131,15 +158,17 @@ If the pipeline is working, the query returns a successful result for the ## HTTPS and private CAs -If either the workload metrics endpoint or the Mimir ingress URL uses TLS signed -by a private CA, install that CA into the machine trust store used by the snap. +If either the workload metrics endpoint or the Mimir ingress URL uses TLS +signed by a private CA, install that CA into the machine trust store used by +the snap. See the CA-handling guidance in [How to integrate COS Lite with uncharmed applications](integrating-cos-lite-with-uncharmed-applications). ## Adapting the scrape config -The example above uses a single static target, but the same pattern works for: +The example above uses a single static target, but the same pattern works +for: - multiple targets in one job - multiple scrape jobs diff --git a/docs/redirects.txt b/docs/redirects.txt index 7b9cc9eb..e5a2b5fb 100644 --- a/docs/redirects.txt +++ b/docs/redirects.txt @@ -12,3 +12,4 @@ "design-goals" "explanation/architecture/design-goals" "juju-topology" "explanation/architecture/juju-topology" "security" "reference/security-hardening-guide" +"tutorial/mimir-on-juju" "how-to/deploy-and-manage/deploy-mimir-on-juju" diff --git a/docs/tutorial/index.md b/docs/tutorial/index.md index 9fe39a95..e0bf314d 100644 --- a/docs/tutorial/index.md +++ b/docs/tutorial/index.md @@ -25,7 +25,6 @@ Production deployment configuration may differ from the setup showcased here. ```{toctree} :maxdepth: 1 -Standalone Mimir on Juju COS on Canonical K8s ``` diff --git a/docs/tutorial/mimir-on-juju.md b/docs/tutorial/mimir-on-juju.md deleted file mode 100644 index 7421aa5f..00000000 --- a/docs/tutorial/mimir-on-juju.md +++ /dev/null @@ -1,192 +0,0 @@ ---- -myst: - html_meta: - description: "Deploy standalone Grafana Mimir on Juju with a small worker topology, S3-backed storage, and a minimal remote_write validation path." ---- - -# Get started with standalone Mimir on Juju - -This tutorial is for readers who already know upstream Grafana Mimir and want -the Juju-native path to a working deployment. - -You will deploy a small standalone Mimir topology on Kubernetes, back it with -S3-compatible object storage, expose it through Traefik, and confirm that it -can ingest metrics over `prometheus_remote_write`. - -```{important} -This guide uses the Juju CLI because it is the clearest way to learn how the -deployment fits together. For repeatable environments and production-oriented -rollouts, prefer the -[Mimir Terraform module](https://github.com/canonical/mimir-operators/tree/main/terraform). -``` - -## Prerequisites - -- Juju 3.6 or later, with a Kubernetes cloud added and a controller already - bootstrapped. -- A Kubernetes cluster ready to host the Mimir model. -- An S3-compatible object store that already exists, including: - - endpoint - - bucket name - - access key - - secret key -- A routable ingress address for Traefik if you want to query Mimir from - outside the cluster. - -If you still need to prepare storage, see -[How to configure object storage for Mimir](../how-to/integrate/configure-object-storage-for-mimir). - -## 1. Create a model - -```bash -juju add-model mimir -juju switch mimir -``` - -## 2. Deploy Mimir and its supporting charms - -Deploy the coordinator, one worker application, Traefik, and an S3 integrator. - -This tutorial uses a compact worker layout for ease of demonstration. The -single `mimir-worker` application runs with `role-all=true`, which keeps the -deployment small while still exercising the coordinator, storage, ingress, and -ingestion flows. - -For higher availability and more production-like scaling, split the worker tier -into dedicated `mimir-write`, `mimir-read`, and `mimir-backend` applications -instead of using this monolithic worker setup. You can then scale each role -independently according to ingestion, query, and storage pressure. - -The tutorial worker also uses `role-query-frontend=true` so queries go through -the same frontend path you will use later. - -```bash -juju deploy mimir-coordinator-k8s mimir --trust - -juju deploy mimir-worker-k8s mimir-worker \ - --trust \ - --config role-all=true \ - --config role-query-frontend=true - -juju deploy s3-integrator mimir-s3 --channel latest/stable --trust -juju deploy traefik-k8s traefik --trust -``` - -## 3. Point the S3 integrator at your object storage - -Replace the placeholder values with your existing S3-compatible storage -details: - -```bash -juju add-secret mimir-s3-credentials \ - access-key= \ - secret-key= - -juju grant-secret mimir-s3-credentials mimir-s3 - -juju config mimir-s3 \ - credentials=mimir-s3-credentials \ - endpoint= \ - bucket= -``` - -If your object store needs additional settings such as a custom CA chain, -region, or path-style addressing, see -[How to configure object storage for Mimir](../how-to/integrate/configure-object-storage-for-mimir). - -## 4. Integrate the deployment - -Connect storage first, then join the worker and ingress relations: - -```bash -juju integrate mimir:s3 mimir-s3:s3-credentials -juju integrate mimir:mimir-cluster mimir-worker:mimir-cluster -juju integrate traefik:ingress mimir:ingress -``` - -Watch the model until all applications settle: - -```bash -juju status --relations --watch=5s -``` - -At this point Mimir should expose at least: - -- `receive-remote-write` for metrics ingestion -- `ingress` for external access -- `self-metrics-endpoint` for scrape-based validation flows - -## 5. Validate metric ingestion with OpenTelemetry Collector - -For a minimal smoke test, deploy: - -- `avalanche-k8s` as a small metrics generator -- `opentelemetry-collector-k8s` as the sender that scrapes Avalanche and - forwards the metrics to Mimir over `prometheus_remote_write` - -```bash -juju deploy opentelemetry-collector-k8s otelcol --trust -juju deploy avalanche-k8s avalanche --trust - -juju integrate avalanche otelcol:metrics-endpoint -juju integrate otelcol:send-remote-write mimir:receive-remote-write -``` - -Wait for the applications to become active: - -```bash -juju status --relations --watch=5s -``` - -## 6. Query Mimir - -Use the Traefik action to get the Mimir URL: - -```bash -juju run traefik/0 show-proxied-endpoints --format=yaml \ - | yq '."traefik/0".results."proxied-endpoints"' \ - | jq -``` - -The output includes a `mimir` entry similar to: - -```json -{ - "mimir": { - "url": "http://10.43.8.34:80/mimir" - } -} -``` - -Save that URL and query Mimir's Prometheus-compatible API: - -```bash -MIMIR_URL=http://10.43.8.34:80/mimir - -curl -sG "$MIMIR_URL/prometheus/api/v1/query" \ - --data-urlencode 'query=count({__name__=~"avalanche_metric_.+"})' \ - | jq -``` - -You should see a successful query response with a value greater than `0`. - -## What you deployed - -This tutorial uses a small Juju-native Mimir deployment: - -- `mimir-coordinator-k8s` provides the public API surface -- `mimir-worker-k8s` runs the actual Mimir workload roles -- `s3-integrator` supplies object-store connection details to Mimir -- `traefik-k8s` gives you a stable URL for querying and ingestion - -That is enough to run Mimir standalone. It also fits naturally into the wider -COS architecture if you later add Grafana, Loki, Alertmanager, or more -specialized telemetry pipelines. - -## Next steps - -- For storage details and production-oriented caveats, see - [How to configure object storage for Mimir](../how-to/integrate/configure-object-storage-for-mimir). -- For more ways to send metrics into Mimir, including cross-model relations and - direct `remote_write` clients, see - [How to send metrics to Mimir](../how-to/integrate/send-metrics-to-mimir).