Skip to content

Troubleshooting

techdox edited this page Jul 14, 2026 · 3 revisions

Troubleshooting

This page covers the common ways Trove can look broken and what to check first.

Dashboard is empty

If OIDC is enabled, use a browser session or TROVE_API_TOKEN for the API checks below. /healthz remains unauthenticated.

Check whether any agents exist:

curl http://SERVER:8080/api/v1/agents

Check whether services exist:

curl http://SERVER:8080/api/v1/services

If agents are empty, no agent has successfully pushed.

If agents exist but services are empty, the agent may be collecting zero hosts/services.

Agent push fails with 401

The Trove token is wrong.

Fix:

docker compose exec server trove-server agent create AGENT_NAME

Update the agent with the new TROVE_TOKEN.

Remember: TROVE_TOKEN is the Trove token, not a Proxmox/Kubernetes/Docker credential.

Agent cannot reach server

From the agent host/container/pod, test:

curl http://SERVER:8080/healthz

If you used localhost, check whether that means the agent itself.

Use a LAN IP, DNS name, service name, or reverse proxy address reachable from the agent.

Proxmox agent shows healthy but no guests

This is usually a Proxmox token permission issue.

Check:

  • token user has PVEAuditor on /
  • token was created with --privsep 0
  • web UI privilege separation was unchecked
  • TROVE_PROXMOX_TOKEN includes user@realm!tokenid=secret

Run:

PVE_TOKEN='trove@pve!trove-agent=PROXMOX_TOKEN_SECRET'
curl -sk -H "Authorization: PVEAPIToken=${PVE_TOKEN}" \
  https://YOUR-PVE-HOST:8006/api2/json/nodes

If it returns {"data":[]}, the token authenticated but has no useful permissions.

Wrong Proxmox image

If the agent card says platform docker when you expected Proxmox, you are running the wrong image.

Use:

ghcr.io/techdox/trove-agent-proxmox:latest

not:

ghcr.io/techdox/trove-agent-docker:latest

Docker agent cannot see containers

Check:

docker logs trove-agent-docker

Confirm the socket mount:

-v /var/run/docker.sock:/var/run/docker.sock:ro

If using a remote Docker endpoint, confirm DOCKER_HOST is correct.

Kubernetes agent running but no data

Check logs:

kubectl logs deploy/trove-agent -n trove

Check:

  • RBAC permissions
  • namespace restrictions
  • server URL from inside the cluster
  • Secret values for TROVE_TOKEN
  • network policies

Alerts not sending

Run:

trove-server alert test

Compose:

docker compose exec server trove-server alert test

If the command says no channels are configured, the variables did not reach the server process.

For Compose, put alert variables under the server service, not the agent.

For systemd, put them in /etc/trove-server.env and restart trove-server.

If only one channel fails, check that channel's endpoint and credentials. Trove records successful per-channel deliveries while retrying only failed channels, so a temporary Discord failure should not duplicate a webhook that already succeeded.

Digest not sending

Required:

  • TROVE_SMTP_HOST
  • TROVE_SMTP_FROM
  • TROVE_SMTP_TO
  • TROVE_DIGEST not off

Also check SMTP port and auth.

Port 465 usually means implicit TLS. Port 587 usually means STARTTLS.

Image freshness is unknown

Possible causes:

  • image has no digest reported by the agent
  • registry requires credentials
  • registry rate limit
  • tag no longer exists
  • registry blocks HEAD/GET in an unusual way
  • freshness disabled

Check server logs for freshness errors.

For private registries, set TROVE_REGISTRY_AUTHS on the server.

Services go stale

A stale service usually means the agent is stale or offline.

Check:

curl http://SERVER:8080/api/v1/agents

Then check that agent's logs and network path.

Database problems

If the server cannot start due to database issues:

  1. stop the server
  2. copy the database somewhere safe before experimenting
  3. inspect logs
  4. restore from backup if needed

Do not delete or overwrite the database unless you are intentionally resetting Trove.

OIDC login or logout loops

Check that TROVE_OIDC_REDIRECT_URL exactly matches the provider callback and that the dashboard root is allowed as a post-logout return URI. For Authentik, both usually need to be allowed:

https://trove.example.com/oauth2/callback
https://trove.example.com/

See Authentication#troubleshooting for discovery, API redirect, and logout checks.

Clone this wiki locally