[VictoriaMetrics] NAS-backed DB migration to LXC 240

[letwir]

Current state: vmctl vm-native is migrating from http://docker.tigris-tailor.ts.net:8428 to VictoriaMetrics on LXC 240.

Tasks:

• Wait for vmctl to exit successfully.
• Verify source and destination readiness.
• Compare total series and representative queries.
• Preserve the NAS source unchanged.

Done when: the first migration is complete and validated.

[letwir]

2026-08-26 JST: vmctl is still active. Source is http://docker.tigris-tailor.ts.net:8428 and destination is LXC 240 at http://127.0.0.1:8428. Last verified destination totalSeries: 85036. Do not start issue #2 until this process exits successfully and destination validation passes.

[letwir]

2026-08-26 JST incident: destination VictoriaMetrics on LXC 240 was OOM-killed at 01:31. systemd restarted it. Storage opens with partsCount=52, rowsCount=1,945,195,065 and about 1.2G on disk, but /api/v1/status/tsdb reports totalSeries=0 and representative historical queries return no samples. Startup deleted unregistered indexdb parts after the unclean shutdown. The original vmctl process is still present after 17+ hours, so the migration is not considered healthy or complete. No source data has been modified. Recovery decision pending; likely stop the stale vmctl, preserve/rename the current destination, apply a memory limit/concurrency-safe configuration, and rerun into a clean destination.

[letwir]

2026-08-26 recovery restart: LXC 240 configured and effective at 4 GiB RAM / 8 GiB swap. VictoriaMetrics now runs from /opt/victoriametrics/db/tsdb with -memory.allowedBytes=2GiB; failed OOM destination was preserved at /var/lib/victoriametrics.failed-20260826-oom. NAS migration restarted as vmctl-nas-migration.service with MemoryMax=1G and concurrency=1. Current verification: service active, destination 11,091 series and growing, TSDB 395 MB; VictoriaMetrics about 1.36 GiB RSS, vmctl about 46 MiB. Source and failed destination were not deleted. Migration is not complete yet.