You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
👋 New to Kubernetes? No problem.
kwatch watches your cluster 24/7. When something fails, it tells you what broke and why — with the error reason, diagnostic hints, logs, and events — straight to your team chat.
✨ 60 seconds to install. No backend. No dashboards. No YAML spaghetti.
🧐 What is kwatch?
kwatch is like a smart friend for your Kubernetes cluster:
💥 Something crashes → you get a message that says why (not just "pod is broken")
🔇 Smart about noise — groups related issues into a single notification, ignores flapping
🧠 Optional AI that reads the logs and tells you what's likely wrong
⚡ Works in under a minute — just one command and a config file
No Prometheus. No Grafana. No 50-step setup. Just alerts that make sense.
Links incidents to root causes — unhealthy nodes, bad rollouts, misconfigured ConfigMaps/Secrets
📊 Mass failure detection
✅ on
Detects when 30%+ of dependents sharing a node/configmap/secret fail simultaneously
✅ TLS is the only one off — everything else just works out of the box.
🧠 Context-aware intelligence
kwatch builds a dependency graph of your cluster from pod informers — mapping pods to their nodes, owners (Deployments/StatefulSets/DaemonSets), Services, PVCs, and referenced ConfigMaps/Secrets. When an incident fires, the insight engine analyzes it against the graph and answers:
What likely caused this? — unhealthy node, failed rollout, or misconfigured resources (ConfigMap, Secret, PVC)
What's the impact? — how many pods, services, or dependents are affected
Recent changes? — correlated changes on the same resource or namespace
The graph is built at startup from the informer cache, rebuilt periodically, and updated incrementally as pods come and go. No configuration needed.
Mass failure detection
The correlation engine periodically scans all active incidents for shared dependencies. If more than 30% of dependents sharing a node, ConfigMap, Secret, or PVC are in failure, a mass failure alert fires. The threshold is dynamic — computed per dependency based on the current scope. Mass failures automatically resolve when the underlying incidents clear.
🤖 AI-powered troubleshooting (optional, off by default)
kwatch ships with built-in AI (runs inside your cluster — zero data leaves). It is disabled by default; set enabled: true to use it:
llm:
enabled: false # ❌ off by default!
When a crash happens, the AI reads the logs and tells you the most likely cause and what to do next. Like having a senior SRE on-call with you.
📌 Architecture note: AI is available for linux/amd64 and linux/arm64 only. It does not support arm/v6 or arm/v7 (the main kwatch image supports all four).
⚙️ Configuration (simple)
🔧 General
Parameter
What it does
maxRecentLogLines
How many log lines to include in alerts (default: 50)
resyncSeconds
Check for problems periodically (0 = only on events, recommended)
workers
How many parallel workers (default: 1, raise for big clusters)
namespaces
🔽 Limit to specific namespaces, or use !kube-system to exclude
reasons
🔽 Only alert on specific reasons, or exclude some with !
ignoreFailedGracefulShutdown
✅ Skip containers killed during graceful shutdown (default: true)
ignoreDisruptionTerminations
✅ Skip pods evicted during node drains (default: true)
runbooks
📚 Add links to your runbooks per error reason
llm.enabled
🤖 AI enrichment (default: false)
containerRestartThreshold
Alert if a container restarts this many times (0 = off)
reportStartupBaseline
📋 Send one startup summary of pre-existing issues (default: true)
🔽 Filter by namespace
# Watch only these namespacesnamespaces:
- default
- production# Or exclude some (can't mix both)namespaces:
- !kube-system
- !monitoring
🔽 Filter by reason
# Only these reasons trigger alertsreasons:
- CrashLoopBackOff
- ImagePullBackOff# Or exclude somereasons:
- !Started
- !Killing
📱 App settings
Parameter
What it does
app.proxyURL
🔗 Proxy for outgoing HTTP requests
app.clusterName
🏷️ Name shown in alerts so you know which cluster
app.disableStartupMessage
Silence the "kwatch is alive" welcome message
app.logFormatter
Log format: text (default) or json
includeEvents
📋 Include K8s events in alerts (default: true)
includeLogs
📋 Include container logs in alerts (default: true)
💓 Health checks
Parameter
What it does
healthCheck.enabled
✅ Health endpoints (default: true)
healthCheck.port
Port to serve health on (default: 8060)
healthCheck.pprof
🔬 Go profiling endpoints (default: false)
healthCheck.diagnostics
🩺 Extra endpoints: /incidents, /test-alert, /deadletters
Endpoints:
GET /healthz — ✅ Liveness
GET /readyz — ✅ Readiness
GET /health — {"status": "ok"}
GET /incidents — 📋 All active incidents (requires diagnostics: true)
POST /test-alert — 📤 Send a test alert (requires diagnostics: true)
GET /deadletters — 💀 Recent delivery failures (requires diagnostics: true)
⏱️ Minutes of unavailability before alerting (default: 2)
📡 DaemonSet Monitor
Parameter
What it does
daemonSetMonitor.enabled
✅ Watch for unavailable DaemonSet pods (default: true)
🧑💼 Job Monitor
Parameter
What it does
jobMonitor.enabled
✅ Watch for failed/suspended Jobs (default: true)
⏰ CronJob Monitor
Parameter
What it does
cronJobMonitor.enabled
✅ Watch for suspended CronJobs or missed schedules (default: true)
📈 HPA Monitor
Parameter
What it does
hpaMonitor.enabled
✅ Watch HPAs stuck at max replicas (default: true)
hpaMonitor.sustainedMinutes
⏱️ How long before alerting (default: 20 min)
💓 Heartbeat Monitor (dead man's switch)
Parameter
What it does
heartbeatMonitor.enabled
Send pings to a health-check URL (default: false)
heartbeatMonitor.interval
⏱️ Seconds between pings (default: 300)
heartbeatMonitor.url
🔗 External URL (e.g. Healthchecks.io)
If kwatch stops or crashes, the external monitor stops getting pings and pages you. 🔔
🔒 TLS Certificate Monitor
Parameter
What it does
tlsMonitor.enabled
🔐 Watch for expiring certs (default: false)
tlsMonitor.threshold
📅 Days before warning (default: 30)
tlsMonitor.criticalThreshold
🚨 Days before critical (default: 3)
🔗 Service Endpoint Monitor
Parameter
What it does
serviceMonitor.enabled
🔗 Watch for services with zero ready endpoints (default: true)
Detects when a Service's backing Endpoints object has zero ready addresses, indicating no healthy pods are available to serve traffic. Includes a 60-second debounce to avoid flapping during rolling updates or brief endpoint transitions.
🧩 Admission Webhook Monitor
Parameter
What it does
admissionWebhookMonitor.enabled
🧩 Watch for webhooks with unreachable backends (default: true)
Monitors MutatingWebhookConfiguration and ValidatingWebhookConfiguration resources. Alerts when a webhook's backing service has no ready endpoints, meaning admission requests may fail or timeout.
🏛️ Control-Plane Monitor
Parameter
What it does
controlPlaneMonitor.enabled
🏛️ Watch for broken control-plane components (default: true)
Detects container issues (CrashLoopBackOff, Error, OOMKilled, etc.) in control-plane pods (kube-apiserver, kube-scheduler, kube-controller-manager, etcd, kube-proxy, coredns). Runs a dedicated sweep at startup to catch pre-existing failures.
🌐 Ingress Backend Monitor
Parameter
What it does
ingressMonitor.enabled
🌐 Watch for ingress backends with no ready endpoints (default: true)
Alerts when an Ingress rule references a backend service that has zero ready endpoints, meaning traffic to that host/path would return an error.
Detects NetworkPolicy resources that deny all ingress traffic (no ingress rules defined). Helps identify policies that may unintentionally block legitimate traffic.
🧩 StatefulSet Monitor
Parameter
What it does
statefulSetMonitor.enabled
✅ Watch for unavailable StatefulSet pods (default: true)
statefulSetMonitor.sustainedMinutes
⏱️ Minutes of unavailability before alerting, plus 15-minute rollout grace (default: 5)
Monitors StatefulSets where readyReplicas < replicas for a sustained period, with a 15-minute grace window during rollouts to avoid alerting mid-update.
🔄 PDB Monitor
Parameter
What it does
pdbMonitor.enabled
✅ Watch for PDBs blocking voluntary disruptions (default: true)
pdbMonitor.sustainedMinutes
⏱️ Minutes of blocking before alerting (default: 5)
Alerts when a PodDisruptionBudget has disruptionsAllowed=0 and currentHealthy < desiredHealthy, meaning voluntary disruptions (rollouts, node drains) are blocked.
🏭 Node Resource Monitor
Parameter
What it does
nodeResourceMonitor.enabled
✅ Check node overcommit levels (default: true)
nodeResourceMonitor.intervalSeconds
⏱️ How often to check (default: 300)
nodeResourceMonitor.cpuWarning
⚠️ CPU overcommit ratio for warning (default: 2.0)
nodeResourceMonitor.cpuCritical
🚨 CPU overcommit ratio for critical (default: 4.0)
nodeResourceMonitor.memWarning
⚠️ Memory overcommit ratio for warning (default: 2.0)
nodeResourceMonitor.memCritical
🚨 Memory overcommit ratio for critical (default: 4.0)
Periodically computes the ratio of pod resource requests vs node allocatable for CPU and memory. Data is purely in-memory — no TSDB or persistent storage needed.
💥 OOM Pattern Monitor
Parameter
What it does
oomMonitor.enabled
✅ Track repeating OOMs (default: true)
oomMonitor.threshold
🔢 OOM count within window to flag (default: 3)
oomMonitor.windowMinutes
⏱️ Sliding window in minutes (default: 60)
Tracks OOMKilled events per container in a sliding window. When the threshold is exceeded, the reason changes from OOMKilled to OOMRepeating with a hint suggesting a potential memory leak.
🎯 Scheduling Delay Diagnostics
Parameter
What it does
scheduleMonitor.enabled
✅ Compute unschedulable delay (default: true)
When a pod is stuck Unschedulable, computes now - PodScheduled.LastTransitionTime and prepends the delay to the hint (e.g., "unschedulable for 5m30s — ...").
⏳ Pending Pod Threshold — alert after N seconds stuck in Pending (default: 300s)
🎯 Severity
Parameter
What it does
severityByOwnerKind
Set severity per resource type, e.g. StatefulSet: "high"
Defaults: StatefulSet → 🔴 high, everything else → 🟡 normal
⏱ Grouping window in seconds (default: 60). Set to 0 to disable.
kwatch groups related incidents by the dimension that best captures each failure type's root cause. For example, OOMKilled and probe failures group by owner+namespace, node conditions group by node (not pod errors on the same node), image pull errors group by image (or globally for rate limits), and CrashLoopBackOff with a matching log signature bridges across owners. Each group notification shows affected pods, owners, nodes, or images depending on scope, with overflow counting above 1,000 entries. After a group notification is sent, the same condition will not re-notify until the underlying incident is resolved and re-occurs, preventing periodic flooding.
📋 CRD — live config changes
Parameter
What it does
crd.enabled
Watch KwatchConfig CRs for live config updates (default: false)
kwatch is not a metrics collector, dashboard, or observability backend.
No TSDB, no dashboards, no log storage, no query language.
kwatch is the alarm — your existing tools are the archive.
Need full observability? Pair kwatch with Prometheus + Grafana for metrics,
or Loki for logs. kwatch handles the one thing a dashboard cannot: telling
you something broke right now. ⏰
