Twitch Crawler

Worker used to collect viewers count from Twitch streams and export them as OpenTelemetry metrics to SigNoz (or any OTLP-compatible backend).

Metrics output is selected by --otel-exporter-endpoint: set it to export over OTLP/HTTP, or leave it unset to print metrics to the console (useful for local development).

Architecture

The project is organized into four modules:

• config — CLI and environment configuration (parsed with clap)
• crawler — Twitch stream fetching and metric recording logic
• telemetry — OpenTelemetry meter provider initialization (OTLP/HTTP or stdout)
• main — Application entry point, measurement loop, and token management

Metrics

Metrics exported:

• twitch.viewers (gauge) — Number of viewers on a Twitch stream

Metric attributes:

• Custom labels via OTEL_ADDITIONAL_LABELS (format: key1=value1,key2=value2)
• stream_id
• game_id
• game_name
• user_id
• user_name

Build

Rust version: 1.97+

cargo build --release

Usage

CLI

Parameters can be passed to binary by environment variables or as command parameters.

twitch-crawler -h

Usage: twitch-crawler [OPTIONS] --twitch-client-id <TWITCH_CLIENT_ID> --twitch-client-secret <TWITCH_CLIENT_SECRET>

Options:
      --otel-additional-labels <OTEL_ADDITIONAL_LABELS>
          Extra labels added to every record, format: key1=value1,key2=value2 [env: OTEL_ADDITIONAL_LABELS=] [default: ""]
      --twitch-client-id <TWITCH_CLIENT_ID>
          Twitch client id [env: TWITCH_CLIENT_ID=]
      --twitch-client-secret <TWITCH_CLIENT_SECRET>
          Twitch client secret [env: TWITCH_CLIENT_SECRET=]
      --game-ids <GAME_IDS>
          Filter streams on game ids (Max value: 100) [env: GAME_IDS=]
      --languages <LANGUAGES>
          Filter streams on languages (Max value: 100) [env: LANGUAGES=]
      --user-logins <USER_LOGINS>
          Filter streams on user logins [env: USER_LOGINS=]
      --minimum-viewers <MINIMUM_VIEWERS>
          Keep only datapoint viewers count superior to the value [env: MINIMUM_VIEWERS=] [default: 0]
      --interval <INTERVAL>
          Interval of seconds between each measurement [env: INTERVAL=] [default: 15]
      --otel-exporter-endpoint <OTEL_EXPORTER_ENDPOINT>
          OTLP metrics endpoint. When set, metrics are exported to this endpoint; when unset, metrics are printed to the console [env: OTEL_EXPORTER_ENDPOINT=]
  -h, --help
          Print help information
  -V, --version
          Print version information

We use env_logger crate to handle logging, if you need to have debug logs you can add environment variable RUST_LOG=twitch_crawler=debug.

SigNoz Cloud

To send metrics to SigNoz Cloud, set the following environment variables:

export OTEL_EXPORTER_ENDPOINT="https://ingest.<region>.signoz.cloud:443/v1/metrics"
export OTEL_EXPORTER_OTLP_METRICS_HEADERS="signoz-ingestion-key=<your-ingestion-key>"
export OTEL_SERVICE_NAME="twitch-crawler"

Docker

export TWITCH_CLIENT_ID=xxx
export TWITCH_CLIENT_SECRET=xxx
export OTEL_EXPORTER_ENDPOINT=https://ingest.us.signoz.cloud:443/v1/metrics
export OTEL_EXPORTER_OTLP_METRICS_HEADERS="signoz-ingestion-key=xxx"

docker-compose up -d

Key changes from previous version

• Warp10 removed — metrics are no longer pushed to Warp10; OpenTelemetry with OTLP/HTTP export replaces it entirely.
• Crate migration — upgraded from twitch_api2 / twitch_oauth2 to twitch_api 0.8 (unified crate).
• OpenTelemetry stack — added opentelemetry 0.32, opentelemetry_sdk, opentelemetry-otlp, and opentelemetry-stdout for metrics instrumentation.
• Modular codebase — logic split into config, crawler, and telemetry modules instead of a single main.rs.
• --event-name removed — replaced by the generic --otel-additional-labels flag to add arbitrary key=value labels.
• Proper error handling — uses anyhow::Result throughout instead of expect()/panic!.
• Docker image — base image updated to rust:1.97-slim-bookworm / debian:bookworm-slim.