Controller integrations: Proxmox, UniFi, Omada, OPNsense, Dockhand with review-first import and scheduled sync - #128
Open
freedbygrace wants to merge 21 commits into
Conversation
Adds the storage and API layer for first-class controller/hypervisor integrations (Proxmox VE, UniFi Network, TP-Link Omada, OPNsense): - New integrationConnections table (schema v36), lab-scoped with ON DELETE CASCADE, holding a per-connection provider, base URL, auth kind, optional auth id, and a single AES-256-GCM encrypted secret via the existing RACKPAD_SECRET_KEY secret-crypto helper (same pattern as SNMPv3 credentials and Docker import tokens). - server/lib/integrations/ with provider registry metadata and a connections store that mirrors snmp-credentials.ts: public rows expose hasSecret only; secrets decrypt server-side through loadIntegrationConnectionSecrets(). - /api/integrations routes: provider metadata plus connection list/create/update/delete. Creating or rotating a secret requires RACKPAD_SECRET_KEY (503 otherwise); writes require lab write access and reads are lab-filtered like SNMP credentials. - Per-connection sync toggles (VLANs / subnets / DHCP) are stored up front so mixed environments can pull L2 from one controller and L3 from another (for example Omada VLANs with networks terminating on OPNsense) without conflicting ownership. - Per-connection verifyTls flag for self-signed homelab controllers. No existing behavior changes; this commit only adds the registry the provider API clients build on in follow-up commits. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Wires integration connections into the same review-first sync flow SNMP inventory sync already uses: - server/lib/integrations/http.ts: a guarded JSON transport for controller APIs. Every request is DNS-resolved through net-guard (so loopback/link-local/metadata targets are rejected), pinned to the resolved address with Host/SNI preserved, size-capped at 8 MB, and supports the per-connection verifyTls opt-out for self-signed homelab controllers. A transport seam mirrors the Docker importer's setDockerHttpJsonFetcherForTests pattern. - server/lib/integrations/inventory.ts: the provider client contract (test + fetchInventory returning a normalized VLAN/subnet/DHCP collection plus a read-only device preview list) and a small registry with a test override seam. - server/lib/integrations/network-sync.ts: bridges provider pulls into the existing snmp-sync engine, so integration applies get the exact same merge/mirror semantics, delete blockers, DHCP preview-only handling, and IPAM safety rules with zero duplicated logic. The connection's per-kind sync toggles filter the collection and surface a warning listing anything skipped. - server/lib/snmp-sync.ts: applySnmpSyncPreview accepts an optional audit context (entity type, action prefix, label) so integration applies audit as IntegrationSync/integration.sync.* while SNMP behavior and audit output stay byte-for-byte identical. - New routes: POST /api/integrations/connections/:id/test (records ok/error status and product info on the connection), .../inventory (pull + preview; refuses disabled connections), and .../apply (admin-only, validates the preview belongs to the connection and its lab, mirror deletes require allowDeletes confirmation). - Tests drive the full pipeline through an injected fake client: status recording, preview counts, sync-toggle filtering, DB writes on apply, audit rows, admin gating, and 501/409 edge cases. The two migration assertions in app.test.ts move to schema version 36. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds the first live provider client. Auth uses Proxmox API tokens
(PVEAPIToken header) — create a read-only token with PVEAuditor on /
and paste user@realm!tokenid as the auth id.
- Test connection reads /version, /nodes, /cluster/status, and
/cluster/resources, reporting product version, cluster name,
node list, and QEMU/LXC counts on the connection card.
- Network inventory maps node bridge/VLAN interface CIDRs and SDN
vnets/subnets (including SDN DHCP ranges) into the shared
VLAN/subnet/DHCP collection, and lists nodes, workloads, and
bridges as read-only device previews. SDN endpoints are optional —
missing permissions or unconfigured SDN degrade to a warning.
- fetchProxmoxStagedInventory() rebuilds the exact JSON payload that
scripts/collect-proxmox.sh produces (schema
rackpad.proxmox.inventory.v1) by walking /nodes/{node}/qemu|lxc
config, status, guest-agent NICs, and live LXC interfaces. That
means a live API pull feeds the existing review-first Proxmox
import wizard unchanged: same host mapping, VM/LXC staging, bridge,
virtual-port, VLAN, and IP-conflict handling as a file upload, and
no VM or container is silently dropped (per-workload request errors
are carried in collectorErrors for the wizard to show).
- New routes: GET /api/integrations/connections/:id/proxmox/nodes
and POST .../proxmox/staged-inventory { node? } for the wizard.
- Unit tests fake the HTTP transport and pin the auth header format,
inventory mapping (bridge CIDR canonicalization, SDN VLAN linking,
DHCP ranges), the staged payload against collector semantics (disk
parsing minus CD-ROMs, NIC model/MAC/VLAN tag parsing, LXC live
IPs, swap/unprivileged flags), node validation, and 401 mapping.
Developed with AI assistance (Claude), reviewed and directed by a
contributor with 15+ years of professional IT experience.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds the OPNsense provider client using API key + secret over HTTP Basic auth (System > Access > Users > API keys; a Viewer-privileged dedicated user is enough for the read-only calls used here). - Test connection reads /api/core/firmware/info for product/version and the diagnostics system information endpoint for the hostname. Because OPNsense 25.7 switched registered API URLs from camelCase to snake_case (and restricted keys can 403 on the unregistered casing), every call retries its snake_case form on 403/404 so both old and new releases work with least-privilege keys. - Network inventory maps: VLAN definitions from /api/interfaces/vlan_settings/get (clean machine values, linked to interfaces via the created vlanif device); interface IPv4 CIDRs from /api/interfaces/overview/interfacesInfo into subnets named by interface description; Kea DHCPv4 subnet pools (both start-end and CIDR pool notations) and Dnsmasq DHCP ranges as preview-only DHCP scopes. ISC dhcpd exposes no settings API, so when it is running a warning says its ranges must be documented manually instead of the preview silently missing them. - Device previews list the firewall itself, VLAN and physical interfaces (MAC, address, status, gateway context), and gateway health from /api/routes/gateway/status. - This is the OPNsense-as-L3 half of mixed environments: pair it with a switch-side controller connection and use the per-connection sync toggles so VLAN definitions come from the switching fabric while routed subnets and DHCP come from the firewall (or vice versa) — the shared preview reconciles both against IPAM by VLAN id and CIDR without duplicate ownership. - Unit tests fake the transport and pin Basic-auth format, the snake_case fallback, VLAN/subnet linking, Kea pool parsing (including CIDR pools to host bounds), Dnsmasq ranges, the ISC warning, gateway previews, and 401 mapping. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds the UniFi provider client with both supported auth generations, selected by the connection's auth kind: - API key (recommended): drives the official Integration API at /proxy/network/integration/v1 with the X-API-Key header, paginating sites and devices. Networks come from the 10.x networks endpoints — the list only carries name/VLAN, so gateway-managed networks fetch their detail for the subnet (hostIpAddress/prefixLength) and DHCP server range. On Network 9.x, where the integration API has no networks endpoint, devices still import and a warning explains that VLANs/networks need Network 10+ or username/password auth. - Username/password: logs into the legacy API, auto-detecting UniFi OS consoles (POST /api/auth/login, TOKEN cookie, /proxy/network/api prefix) versus classic software controllers (POST /api/login, unifises cookie, /api prefix). Pulls /self/sites, stat/sysinfo, stat/device, and rest/networkconf with read-only GETs, so no CSRF handling is needed and a view-only local admin account is enough. - Both paths normalize into the shared inventory: corporate/vlan-only networks become VLANs, ip_subnet gateway addresses canonicalize to network CIDRs, dhcpd ranges become preview-only DHCP scopes, and WAN/VPN purposes are excluded so IPAM only sees LAN networks. Devices map usw/uap/ugw/udm/uxg types (or official feature flags) to switch / access-point / gateway previews with firmware context. - The site selector honors the connection's optional site reference, matching id, internal reference key, or display name, and reports the available sites when the reference does not match. - Unit tests fake the transport for all four paths: official API (pagination, header, network detail + DHCP mapping, 9.x warning), UniFi OS cookie flow, classic-controller fallback cookies, and credential rejection for both modes. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds the Omada provider client using the controller Open API with the OAuth2 client-credentials flow (Settings > Platform Integration > Open API; create the app with the Viewer role for read-only access). - Auth discovers the omadacId from the unauthenticated /api/info endpoint, posts the client credentials to /openapi/authorize/token?grant_type=client_credentials, and sends the documented "Authorization: AccessToken=..." header (not Bearer) on every call. Non-zero errorCode envelopes surface the controller's own message, so bad client secrets read clearly. - Site and device grids paginate with the required page/pageSize params until totalRows is reached. Devices map the switch / ap / gateway types and numeric status codes into read-only previews with model, MAC, IP, and firmware context. - LAN networks try the v3, v2, then v1 lan-networks schema, because the endpoint family arrived with controller 5.15.x and the schema version varies by firmware; pre-5.15 controllers degrade to a device-only pull plus an explanatory warning. Networks map VLAN ids and gatewaySubnet CIDRs into the shared collection, and enabled DHCP servers contribute their ipRangePool ranges (v1 single-range fields as fallback) as preview-only DHCP scopes. - This is the L2/VLAN half of the mixed Omada + OPNsense scenario: keep syncVlans on here and syncSubnets/syncDhcp on the firewall connection (or any mix) and the shared preview reconciles both sources by VLAN id and CIDR without duplicate ownership. - Unit tests fake the transport and pin the token request shape (grant_type query + JSON body), the AccessToken header, grid pagination fields, device/status mapping, the v3->v2 fallback, the pre-5.15 warning, and error-envelope surfacing. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds the user-facing half of the controller integrations, following the existing Docker import panel and SNMP sync panel patterns: - New Integrations card in Imports with per-provider add buttons and hand-drawn, trademark-neutral provider marks (hypervisor hexagon, wireless rings, mesh, shield) in the lucide stroke style. - Connection management: create/edit forms adapt their auth fields to the provider (API token, API key or username/password, client credentials, key/secret), passwords are write-only (leave blank to keep the stored secret), and each connection carries TLS verification, an optional site, and VLAN/subnet/DHCP pull toggles with inline guidance for mixed switch-plus-firewall environments. - Connection rows show live status (untested/connected/error with the server-recorded error), the product and version from the last test, and actions to test, pull inventory, edit, and delete. - Pull inventory renders the shared preview: VLAN and subnet diffs with create/update/delete badges and blocked-delete reasons, DHCP ranges (preview-only), controller warnings, and a read-only device table (name, kind, model, MAC, IP, status). Admins can apply with merge or mirror policy — mirror deletes require the explicit allow-deletes confirmation — and the store reloads after an apply. Editors see the preview but not the apply action. - Proxmox connections add a Stage import action that pulls the live node inventory (with a node picker for clusters) and stages it into the existing review-first Proxmox wizard on the same page, exactly like uploading a collector file. - Typed API client methods and shared types for connections, provider info, test results, inventory responses, and Proxmox nodes. - All new strings are localized in every supported locale; "Bridge" stays English for de/it (allowlisted) to match Proxmox's own UI terminology in those languages. Validated with lint, client tests, i18n checks, bundle budget (201.8 KB gzip of 300 KB), and full client+server builds. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- New docs/INTEGRATIONS.md covering requirements (RACKPAD_SECRET_KEY, LAN reachability, self-signed TLS), step-by-step credential setup for Proxmox VE (PVEAuditor API token), UniFi (API key vs view-only local admin, version caveats), Omada (Open API client-credentials app with the Viewer role, OC200 limitation), and OPNsense (API key/secret, ISC dhcpd range limitation), plus the mixed switch-plus-firewall pull-toggle pattern and the safety model (merge/mirror, delete blockers, DHCP preview-only, admin-only apply, audit trail). - README: integrations bullet in the feature list, quick link, and feature-guide entry. - .env.example: RACKPAD_SECRET_KEY comment now mentions integration credentials alongside SNMP secrets. - CHANGELOG: Unreleased entry with user-facing test notes for each provider and the mixed-environment scenario. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every integration connection already carries a per-connection "Verify TLS certificate" toggle honored by all four provider clients, and HTTPS monitors have per-target ignoreTlsErrors. This closes the remaining gap so the skip-certificate-validation option is available across all outbound connections: Docker/Portainer imports. - dockerImportSources gains a verifyTls column (schema v37, default on) exposed on the source rows. The v34 legacy-migration fixture in app.test.ts now rebuilds the docker tables at their true v34 shape so the new ALTER TABLE migration is exercised against a real pre-upgrade schema. - fetchDockerContainersPreview and the HTTPS fetcher accept an optional verifyTls flag (rejectUnauthorized stays on unless the caller opts out); the injectable test fetcher signature gains the same optional argument, so existing seams keep working. - /docker/preview and /docker/import accept verifyTls and persist it on the upserted source; PATCH /docker/sources/:id now updates enabled and/or verifyTls independently. The background status sync loop reuses each source's stored preference. - The Docker panel shows the verify checkbox in HTTP/Portainer mode and a per-source toggle for saved HTTPS sources, reusing the already-localized "Verify TLS certificate" string, so no new translation keys were needed. - New route-level tests cover the default-on behavior, opt-out propagation to the fetcher, persistence on import, independent PATCH updates, and the sync loop honoring the stored flag. Verification remains enabled by default everywhere; skipping it is an explicit per-connection/per-source choice for self-signed homelab endpoints. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Until now every integration action was manual. This adds the same kind of background freshness the Docker status sync already provides, while keeping inventory strictly review-first: - server/lib/integrations/status-sync.ts re-runs each enabled connection's lightweight test call, updating lastStatus, the product/version summary, and lastError. Disabled connections and connections without a stored secret or provider client are skipped; per-connection failures are recorded on the row and never abort the sweep. - Wired into server startup like the other loops, guarded against overlapping runs, unref'd, and stopped on shutdown. Interval comes from INTEGRATION_STATUS_SYNC_INTERVAL_MS (default 300000 ms to match the Docker status loop; 0 disables), documented in .env.example. - Inventory pull/apply intentionally stays manual: the docs now spell out what is automatic (connection status, Docker container status) versus manual by design (VLAN/subnet/DHCP preview and apply), so nothing writes to IPAM on a timer. - Tests cover the sweep: ok/error statuses recorded per connection, error aggregation, and disabled connections being skipped. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds Dockhand (github.com/Finsys/dockhand) as the fifth provider, in the same connection/test/pull shape as the controller integrations. - Auth uses Dockhand API tokens (Profile > API tokens) sent as "Authorization: Bearer dh_...". Because a Dockhand token is a single secret, the api-key auth kind now skips the auth-id requirement for every provider that uses it (previously special-cased to UniFi). 401/403 map to a clear token error and the 429 auth rate limit is surfaced with its retry guidance. - Test connection lists /api/environments and reads the app version from the Prometheus dockhand_build_info metric — the only place Dockhand exposes it — reporting environment count and names. - Inventory pull walks /api/dashboard/stats and, per online environment, /api/containers?env=N&all=true and /api/networks?env=N: each environment becomes a host preview (online state, connection type, running/total containers, stack count), each container a preview with image, first network IP, state plus health, and its compose stack, and each Docker network a bridge preview with driver, IPAM subnets, and member count. Offline environments are skipped with a warning instead of being mistaken for empty (Dockhand returns 200 [] on Docker connection failures), and container plumbing is deliberately never fed into IPAM — the collection stays empty with an explanatory note in the preview. - The optional site field acts as an environment filter (name or id, default all environments) with its own localized label; unknown references list the available environments. - New hand-drawn container-and-hook mark in the shared icon set, HTTP port-3000 URL placeholder, and updated panel intro copy; both new strings are translated across all 23 locales and the old intro key is retired everywhere. - Docs cover token creation, the environment filter, the IPAM stance, and the free-edition caveat that any valid token has admin scope; the background status-refresh loop picks Dockhand up automatically. - Unit tests fake the transport and pin the Bearer header, metrics version parsing, environment/container/network mapping, the offline-environment guard (no container queries for offline environments), the environment filter, and token rejection. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds per-connection scheduled inventory sync to the integrations, split across a new tabbed layout (Connections / Auto-sync) so scheduling gets its own space instead of crowding the connection list. - Schema v38 adds autoSync columns to integrationConnections: enabled (off by default — auto-sync is strictly opt-in), mode, cron, target lab ids, failure count, backoff pause, and last-run status/message. - Scheduling accepts basic selectors first (every 15/30 minutes, hourly, every 6 hours, daily, weekly) with custom cron as the advanced option; both store a five-field cron expression parsed by a small dependency-free parser (lists, ranges, steps, the classic day-of-month/day-of-week OR rule) that rejects invalid input at save time. The scheduler ticks once a minute and scans the minutes since the previous tick so short stalls cannot skip a run. - Modes: merge adds missing VLANs/subnets only; overwrite adds and updates to match the controller but never deletes (removals remain a manual, confirmed decision); skip computes the mirror diff and records drift without writing anything. - Multi-lab: a checkbox multi-select of target labs lets one connection populate several labs with the same controller data, defaulting to the connection's own lab; vanished labs are skipped and reported rather than failing the run. - Stability: runs are sequential, overlapping ticks are skipped, and consecutive failures back off exponentially (5m doubling to a 6h cap) so an unreachable controller cannot pile up work or hammer the network. Reconfiguring the schedule clears the backoff. - Errors surface without taking over: each connection row in the Auto-sync tab shows a status badge (synced/drift/error/backing off), the last run time, and the exact failure message inline; applies are audited as integration.sync.* under the integration-auto-sync actor. - Configuration is admin-only (editors keep connection management but scheduled writes bypass per-run review); a Run now action executes the configured sync immediately. All 29 new strings are translated across the 23 locales. - Tests cover the cron parser, admin gating and validation (bad cron, missing schedule, unknown labs), multi-lab population, per-mode behavior including drift detection, exponential backoff with recovery, and the due-schedule scanner's catch-up window. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Applies a full usability pass on the integrations flow based on hands-on feedback, plus two functional fixes: - The Imports page now has horizontal Imports / Integrations tabs: collector uploads, NetBox, and Docker imports stay exactly where they were on the first tab, and the integrations panel gets its own space. Staging a Proxmox pull switches back to the Imports tab where the wizard lives. - Prose is gone in favor of signposts: the panel intro paragraph and the "Mixed setups" hint are removed; provider add buttons, action buttons, and every checkbox carry concise hover descriptions, and the connection form shows a one-line provider summary (no secret key jargon). - Pull checkboxes now correlate to each integration: Proxmox shows "SDN VLANs" (hover explains overlay fabric may not match physical switch VLANs), "Bridge and SDN subnets", "SDN DHCP ranges"; UniFi and Omada show network/LAN wording; OPNsense shows interface and Kea/Dnsmasq wording; Dockhand hides them since nothing feeds IPAM. "(preview only)" moved from labels into hover text. - New Test & discover flow: schema v39 adds scopeRefs, the /api/integrations/discover-scopes route verifies inline or stored credentials and lists sites (UniFi/Omada), cluster nodes (Proxmox), or Docker environments (Dockhand), and the form stores the checked scopes. Every provider pulls exactly the selected scopes, labelling devices and networks by site/node on multi-scope pulls; the legacy single site field remains honored as a fallback. - Inventory previews open in a modal with one tab per object type (VLANs, subnets, DHCP, devices) with counts in the tab labels and the policy/apply controls in the footer, replacing the in-page scroll. - VLAN association: when a pull reports a VLAN id for a subnet that already exists without a VLAN link (the OPNsense/pfSense case), the merge preview now shows a link-only update and apply associates the subnet with the VLAN — names and other fields are never touched, audited as integration.sync.subnet.link. - Omada device hardening: device kinds match case-insensitively across type/deviceType/deviceCategory so switches are never dropped, and an empty per-site device list falls back to the controller-wide endpoint with an explanatory warning. - 45 new UI strings translated across all 23 locales; 9 obsolete keys removed everywhere. Tests cover the discover-scopes route, scope storage, and all provider suites against the new multi-scope behavior. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ules Controller integrations previously previewed devices read-only; switches, gateways, and APs never became Rackpad records, and auto-sync was limited to one schedule per connection. This change makes device inventory real and scheduling flexible, and clears up two UX problems on the panel. Device and WiFi import (server/lib/integrations/device-sync.ts): - UniFi and Omada pulls now build importable device records for switches, gateways, and access points, including the full switch port list (name, RJ45/SFP/SFP+ media, speed, link state) fetched from the per-device detail endpoints in both official-API and legacy modes. OPNsense contributes the firewall itself. - Applying creates devices as loose gear (placement "room", no rack) so physical location is never guessed, with ports, model, MAC, IP, serial, firmware, and online status. Matching is merge-only: MAC address first, then hostname/display name; existing records are never modified. - Enabling the SSID pull upserts a WiFi controller per connection, imports SSIDs (VLAN-linked when ids match), and links imported APs to the controller. Everything is audited (integration.sync.device.create, integration.sync.wifi.*). - Echoed import payloads are re-validated server-side with size caps (500 devices / 128 ports / 200 SSIDs) before applying. Multiple schedules per connection (server/lib/integrations/schedules.ts): - New integrationSyncSchedules table (migration v40) with a data migration that converts the previous per-connection auto-sync columns into a "Default schedule" row, so existing setups keep running unchanged. - Each schedule has its own name, cadence (presets or five-field cron), merge/overwrite/skip mode, and multi-lab target list; failure backoff (5m doubling, capped at 6h) is now tracked per schedule. Scheduled runs also import new devices/SSIDs when the connection's pulls enable it. - CRUD + run-now routes are admin-gated; the minute-scan loop is unchanged and runs strictly sequentially. Integrations panel: - The preview modal gains an Import tab showing create vs. already-tracked for devices (with port counts) and SSIDs, applied via an explicit "Import devices" button; per-provider Devices/SSID pull toggles with hover descriptions; discovered sites/nodes/environments moved into a checkbox dropdown matching the target-labs control; the connections list gets an "Existing connections" heading and a provider badge per row; the Auto-sync tab lists and edits any number of schedules per connection. - Fixed Proxmox staging silently failing: the wizard's parse result now flows back to the panel, so a payload the wizard rejects shows an error instead of a false success, and node lists are prefetched so multi-node clusters show the node selector up front instead of requiring a second click. Migration v40 adds syncDevices/syncWifi columns (default on) plus the schedules table. New strings are translated across all locales; docs and changelog updated. Server suite, client tests, lint, check:i18n, and the production build all pass. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The single "switches, gateways, APs" pull checkbox hid choices the controllers can actually delineate, imported switch ports carried no VLAN information, and the preview modal showed both apply buttons on every tab. Per-category device toggles (migration v41): - syncDevices splits into syncSwitches / syncGateways / syncAccessPoints, seeded from the previous setting so existing connections keep importing exactly what they did (the old column stays dormant). UniFi and Omada report device type reliably, so each category gets its own checkbox with its own hover hint; OPNsense keeps its single "Firewall device" checkbox (the firewall follows the gateway toggle internally). - Filtering is enforced server-side in one place (filterImportableDevicesForConnection) and applied to the inventory preview, the apply-devices action (echoed payloads are re-filtered against the stored toggles), and scheduled auto-sync runs. Switch port VLAN mapping: - IntegrationPortSpec gains mode / untaggedVlanNumber / taggedVlanNumbers, and the device import resolves them against the lab's VLANs by number: access ports land with their untagged VLAN linked (ports.vlanId), trunks land with mode "trunk" and the carried VLANs in allowedVlanIds. VLANs the lab does not have are dropped silently, so applying the network preview first links everything. - UniFi (legacy API): rest/portconf profiles and per-device port_overrides are resolved per port — forward "native" is an access port on the profile's native network, "all" is a trunk carrying every site VLAN, "customize" is a trunk with the explicit tagged list. Networks and profiles are prefetched per site so the device walk stays one pass. The official integration API exposes VLAN config only on newer Network versions; it is read defensively (native VLAN as an access port). - Omada: the switch portList PVID reads as an access port; the built-in "All" profile reads as a trunk carrying every site VLAN. Preview modal: - The footer is now context-aware: VLANs/Subnets/DHCP tabs show the policy picker and "Apply networks", the Devices/Import tabs show "Import devices" — instead of every control on every tab. Also fixes wrong-language machine translations for the device-type labels the panel surfaces (e.g. Chinese "网关" in the Japanese locale, German "Tor" for gateway); new strings are translated across all locales, and docs/changelog updated. Schema tests cover the v41 migration; the device import test asserts an access port links its VLAN and a trunk stores its tagged list while unknown VLANs are dropped. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The separate Auto-sync tab listed every connection a second time and put schedule management two navigation steps away from the connection it belongs to. Auto-sync now lives on the connection itself, structured the way you reason about it: connection > what to sync > when to sync > where to sync to. - Each connection row gains an Auto-sync expander (with its schedule count). Expanded, it shows "What to sync" — the connection's pull toggles as inline checkboxes that save immediately — followed by "When to sync" with the connection's schedules and Add schedule, where each schedule's editor ends in a "Where to sync to" labs multi-select. - The schedule editor's labels follow the same flow (When to sync for the cadence preset, Where to sync to for the target labs); the connection form groups its pull checkboxes under a "What to sync" heading so creation and later editing read the same way. - The Connections/Auto-sync tab pair is gone; the panel is a single flow again. No behavior changes server-side — schedules, modes, backoff, and multi-lab targeting are untouched. New step labels are translated across all locales; obsolete strings removed; docs and changelog updated. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Proxmox previewed its inventory but imported nothing outside the wizard, staging was limited to one node picked from a plain dropdown, the preview dialog showed empty tabs for object types a product cannot deliver, and the auto-sync explainer repeated under every expanded connection. Proxmox import from the host down (migration v42): - New Hosts and VMs & containers pull checkboxes (syncHosts/syncGuests columns, default on). Pulls now return the selected nodes as importable server devices and their QEMU VMs and LXC containers as vm/container devices with running state, flowing through the same merge-only Import tab as the other providers. Cluster resources carry no NIC/IP detail, so the hints point at Stage import for full fidelity. - The importable device type union gains "vm" and "container", matching the device types the Proxmox wizard already creates, so re-imports and wizard imports match each other by hostname. Multi-node staging: - The staged-inventory endpoint accepts a nodes array (single-node body stays supported). The wizard payload models one host, so the first selected node provides the host summary while workloads come from every selected node; bridges are deduplicated by interface name (vmbr0 across cluster nodes) so guest adapters keep matching their virtual switch. - The per-connection node picker is now a checkbox multi-select defaulting to all nodes. Select all everywhere: - The panel's checkbox dropdowns (target labs, discovered sites / cluster nodes / environments, staging nodes) share one popover control with a Select all row. Preview dialog and copy: - Only tabs with data render (Devices stays as the fallback), so a container platform no longer shows empty VLANs/Subnets/DHCP tabs, and the active tab follows what the pull actually returned. - The auto-sync explainer now appears once above the connections list; the per-connection copies and the repeated "No schedules yet" line are gone. New strings are translated across all locales; docs and changelog updated; migration and integration suites extended accordingly. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…bnets
The preview dialog and the schedules spoke different mode languages
(merge/mirror with an extra delete checkbox versus merge/overwrite/skip
with parenthetical labels), and imported devices sat next to IPAM without
touching it.
One mode vocabulary everywhere (migration v43):
- Mirror, Merge, Skip — plain labels, one hover that explains all three:
Merge only adds missing records; Skip also updates existing records but
never deletes anything; Mirror deletes destination records that are
gone from the source. Stored schedule modes migrate ("overwrite"
becomes "skip", the old drift-only "skip" becomes "merge", the least
intrusive mode that still writes).
- Mirror semantics are per managed object type: a disabled pull option
now drops that type from the diff entirely instead of reading an empty
source list as "delete everything in the destination". Referenced
VLANs/subnets stay protected by the engine, and devices, SSIDs, and IP
assignments are merge-only in every mode.
- Scheduled mirrors can therefore prune stale records automatically; run
messages report removals. The old drift-report mode is gone — the
preview dialog is the review tool.
- The preview dialog drops the separate allow-deletes checkbox (Mirror
means deletes, with the same protections) and re-pulls automatically
when the mode changes so the diff always matches the selected mode.
Interconnecting the import:
- A device whose IP falls inside a subnet the lab already tracks is
linked as an IP assignment on that subnet during device import —
for newly created and matched devices alike, never touching an address
that is already assigned. Audited as integration.sync.ip.create and
reported in the import result. Combined with the existing links (ports
to VLANs, trunks to tagged sets, SSIDs to VLANs, APs to controllers,
subnets to VLANs), a networks-then-devices pull now comes out
interconnected rather than side by side.
Docs describe the new mode table and the safety model; the changelog and
all locales are updated; the auto-sync suite covers skip-updates,
mirror-deletes, protection of in-source records, and IP-assignment
idempotency.
Developed with AI assistance (Claude), reviewed and directed by a
contributor with 15+ years of professional IT experience.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Stage import button hand-off to the wizard was a second path with its own node picker, and the regular pull still imported Proxmox guests as bare name/state records — no VMs with NICs, no virtual switches. One path now carries everything. Proxmox host-down import: - The pull walks each selected node's guests exactly like the offline collector: per-guest config, status, QEMU guest agent addresses, and live LXC interfaces. VMs and containers import as virtual devices attached under their node (placement "virtual", parentDeviceId), each virtual NIC as a port carrying MAC, access/trunk VLAN tag, and the link to its virtual switch; NIC addresses that land in known subnets become IP assignments on the port. Templates are skipped. - Node bridges import as Rackpad virtual switches on the host device (external when they have member ports, internal otherwise; VLAN subinterfaces like vmbr0.20 ride on the bridge rather than becoming their own switch). The Import tab lists them with create/already- tracked states and the apply upserts them idempotently by host+name. - Import ordering is host → virtual switches → guests inside one transaction, so parent and vswitch links always resolve in a single Import devices click. - Stage import is gone: the button, the per-row node picker, the staged-inventory and node-list endpoints, and the wizard bridge. The connection's node scope (Test & discover) decides what a pull covers, and the offline collector upload on the Imports tab stays for hosts Rackpad cannot reach. MAC normalization: - Every MAC an integration writes — devices and ports, all providers — is canonicalized to uppercase colon-separated form (AA:BB:CC:DD:EE:FF). Omada's dashed MACs, UniFi's lowercase, and Proxmox's raw hex all come out identical, so cross-source matching and display stay consistent. Tests cover the rich guest inventory (NIC/VLAN/vswitch/IP mapping from the fake cluster), the phased apply (guest under host, NIC on vswitch, VLAN link, port-level IP assignment, idempotent vswitch upsert), and MAC canonicalization. Docs, changelog, and all locales updated. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Verified against a live two-node OVS cluster: /network listed only OVSPort rows (the bridges existed solely as ovs_bridge references), so no virtual switches imported, and the guest endpoints returned empty 200s because the privilege-separated API token had no ACL of its own — Proxmox filters listings by permission instead of erroring, which made the pull look broken with no explanation. - parseNodeNetwork captures ovs_bridge/ovs_ports, and missing OVS bridges are synthesized from their member ports (member list filled in, external kind, active when any member is). VLAN subinterfaces still ride on their bridge. - A pull that sees zero guests across cluster/resources and every node's qemu/lxc lists now pushes a warning telling the operator to grant the token PVEAuditor on / with Propagate. - Docs call out the privilege-separation trap in the Proxmox setup steps. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dark and light captures of the integrations panel (connections with what/when/where auto-sync) and the review-first inventory preview importing Proxmox hosts, guests, and virtual switches, taken against a seeded demo environment with fake controllers. Added to the README gallery alongside the existing workspace shots. Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Controller API integrations: Proxmox VE, UniFi Network, TP-Link Omada, OPNsense, and Dockhand
This adds live controller integrations to the Imports workspace: Rackpad connects to the APIs homelabs actually run, pulls their inventory, and interconnects it — physical switches with their ports and VLANs, virtual hosts with their virtual switches and guests, routers/firewalls with networks and DHCP — so the lab model makes sense out of the box and the operator can take it from there.
Everything stays review-first: a pull opens a preview diff and nothing is written until an administrator applies it.
Screenshots
Connections carry their own pull toggles and sync schedules (connection → what to sync → when to sync → where to sync to):
The preview dialog shows one tab per object type the product actually returned, and the Import tab is merge-only — create vs. already tracked, applied with one click:
Light-theme captures are in
docs/screenshots/and the README gallery.What's included
Connections
RACKPAD_SECRET_KEY(same AES-256-GCM path as SNMPv3 credentials — secrets never leave the server).INTEGRATION_STATUS_SYNC_INTERVAL_MS, default 5 min).Networks (IPAM)
Devices, ports, and WiFi
AA:BB:CC:DD:EE:FF.Scheduled auto-sync
Provider quirks handled
omadacIddiscovery, lan-networks v3→v2→v1 fallback, controller-wide device fallback.Safety and compatibility
integration.sync.*); apply/import/schedule writes are admin-gated; editors can connect/test/preview; viewers are read-only.Testing
npm run test:server: integration suites cover every provider client against transport fakes (auth flows, fallbacks, port/VLAN mapping, OVS synthesis), the sync engines (modes, backoff, scheduler windows, device/WiFi/IP import idempotency), migrations against legacy-shaped databases, and route-level auth.npm run check:i18n,npm run lint,npm run test:client, and the production build are all green; screenshots were taken against a seeded demo environment backed by fake controller APIs.Developed with AI assistance (Claude), reviewed and directed by a contributor with 15+ years of professional IT experience who runs these controllers. Happy to split, adjust scope, or rework anything to fit the project's direction — the commit history is deliberately granular (one feature area per commit) to make review manageable.
🤖 Generated with Claude Code