[Feature] Community Plugin Preview - Storage Assistant — PVE/NFS and PBS/iSCSI safety model

[mdobprv-lab]

Got an idea? Nice! 💡 We love hearing what features would make PegaProx better for you.

A friendly reminder: PegaProx is built by volunteers in our spare time. We can't promise timelines or guarantee every feature will be implemented — but every suggestion helps shape the roadmap!

Want to help move things faster?

• 💖 Become a Sponsor — sponsored features get prioritized
• ⭐ Star the project — helps with visibility
• 🤝 Even better: submit a PR with your idea!

---

Describe the feature

I am sharing an early community preview of PegaProx Storage Assistant:

• Repository: https://github.com/mdobprv-lab/pegaprox-storage-assistant
• Pre-release: https://github.com/mdobprv-lab/pegaprox-storage-assistant/releases/tag/v0.1.1

This is not a marketplace submission yet. I am publishing the architecture early to identify integration pitfalls before implementing remote storage operations.

The plugin deliberately separates two storage workflows:

• PVE / NFS — cluster-wide technical content only: iso, vztmpl, snippets and import. VM disks and backups are explicitly excluded.
• PBS / iSCSI — one dedicated LUN assigned to one PBS, intended to be formatted with XFS/ext4 and registered as a PBS datastore.

The design is NAS-vendor-neutral and supports definitions for multiple NAS devices and network locations. It is not intended to duplicate vendor-specific storage lifecycle plugins such as the NetApp ONTAP integration described in #443.

Version 0.1.1 is definition-only:

• remote execution is hard-disabled;
• the plugin does not discover, connect, mount, format or register storage;
• NFS paths, iSCSI identities and ownership collisions are validated;
• one-LUN/one-PBS ownership is enforced in the registry;
• credentials and CHAP secrets are not stored;
• definition changes use PegaProx RBAC checks and audit logging;
• English and Polish catalogs are included;
• Modern, Corporate Dark/Light and Cloud Dark/Light layouts were verified;
• 29 automated tests pass in CI.

The plugin has been tested with PegaProx 1.0.2 using the official Docker deployment model.

Before implementing discovery and execution, I would appreciate guidance on the preferred PegaProx integration points:

1. How should a plugin reuse PegaProx-managed PVE and PBS connections without copying credentials into plugin files?
2. Is there a canonical background-task API for progress reporting, cancellation and post-operation status?
3. Which RBAC permissions and audit interfaces should plugin-initiated host operations use?
4. What is the preferred way to resolve and target one PVE node, an entire PVE cluster or one PBS instance?
5. Is there a supported navigation/context model for a global plugin that also performs PBS-oriented operations?
6. Are any relevant plugin APIs or conventions expected to change soon?

The planned execution model is explicit plan → apply → verify → rollback, with typed confirmation before formatting or signature removal and stable device matching by verified IQN, LUN and WWID rather than /dev/sdX.

Any warning about architectural dead ends at this stage would be especially valuable and could prevent significant rework later. There is no expectation of immediate review or marketplace inclusion.

Use Case

PVE clusters often need shared technical storage for ISO images, container templates, snippets and imports without filling local filesystems on individual hosts. NFS is appropriate here because the same technical resource can be available across the cluster.

PBS has a different storage model. A dedicated iSCSI LUN should have one unambiguous owner, be presented only to the selected PBS instance, use a verified filesystem and then be registered as a PBS datastore for VM backups.

The plugin is intended to guide administrators through these distinct workflows while preventing accidental role mixing, duplicate LUN ownership, unsafe device selection and premature destructive operations.

The main architectural requirement is to reuse connection and authorization mechanisms already managed by PegaProx rather than introducing another credential store inside the plugin.

Alternatives considered

• Manual configuration on NAS, PVE and PBS — works, but provides no shared validation, ownership registry, safety review or consistent audit trail.
• Custom SSH scripts — flexible, but difficult to integrate safely with PegaProx RBAC, background tasks, cancellation and auditing.
• Storing separate credentials in plugin configuration — rejected because it duplicates sensitive data and creates an unnecessary security boundary.
• Using iSCSI directly as general PVE storage — rejected for this project because the PVE workflow is intentionally limited to cluster-wide technical NFS content.
• Using one shared iSCSI LUN on multiple PBS/PVE systems — rejected because it creates ambiguous ownership and filesystem-corruption risk.
• Vendor-specific storage plugins — useful for storage lifecycle management, but they do not replace this NAS-vendor-neutral PVE/PBS consumption and safety workflow.

How important is this for you?

Would significantly improve my workflow and prevent architectural rework.

The current definition-only preview already covers validation and persistence. Guidance from the PegaProx maintainers before implementing discovery, background jobs and remote execution would help ensure that the next milestones use supported APIs and conventions.

This is not blocking my current use of PegaProx, and I am not requesting a delivery timeline from the core team.

Checklist

• I have searched existing issues and discussions to make sure this hasn't been requested before

[mkellermann97]

This is a genuinely thoughtful preview — the definition-only, remote-exec-hard-disabled first cut plus the one-LUN/one-PBS ownership model are exactly the right way to de-risk it before touching hosts. Happy to point you at the integration seams:

1. Reusing PegaProx-managed connections (no cred copying). from pegaprox.globals import cluster_managers, pbs_managers gives you the live managers keyed by id: cluster_managers[cluster_id] is a connected PegaProxManager (use mgr._api_get / mgr._api_post for the PVE API, and its node-SSH helpers for on-host commands), and pbs_managers[pbs_id] is the PBS side. The client_portal plugin is the cleanest reference for this pattern. For host-level storage ops specifically, lean on the manager's existing SSH path rather than opening your own connection — you inherit the stored creds and the TOFU host-key verification instead of re-implementing them.

2. Background tasks / progress. There isn't a single formal task framework yet, but the convention is: run long work in a daemon thread and stream progress to the UI via broadcast_sse(update_type, data, cluster_id=...) from pegaprox.utils.realtime (site-recovery / migrations do this through a _broadcast_progress wrapper). For waiting on PVE-side tasks the manager exposes _wait_for_task(...). Honest caveat: there's no first-class cancellation API — you'd track your own cancel flag and check it between steps.

3. RBAC + audit. Decorate routes with @require_auth(perms=[...]) (pegaprox.utils.auth), gate per-object actions with has_permission / user_can_access_vm (pegaprox.utils.rbac), and write log_audit(user, action, detail) (pegaprox.utils.audit) for every state-changing op. plugins/proxmox-ha and plugins/client_portal show all three together — good to mirror their shape.

Given host storage operations are inherently destructive, keeping execution behind an explicit per-op RBAC perm + an audit line (which you're already planning) is the right call. Looking forward to where this goes — shout here if any of those seams are unclear.

[mdobprv-lab]

Thank you — this is exactly the guidance I hoped to receive before starting the discovery and execution layers.

Based on your response, the next milestone will remain read-only and will:

• reuse PegaProx-managed PVE/PBS connections and the existing node SSH path;
• avoid introducing a separate credential or SSH configuration store;
• keep all PegaProx-specific integration behind a small compatibility adapter;
• use broadcast_sse(...) for progress reporting and plugin-owned cancellation checkpoints;
• mirror proxmox-ha and client_portal for authorization, object-level access checks and audit logging.

I reviewed the current helpers after your comment. Before the later execution-capable milestone, I would appreciate clarification on three details:

1. For PVE, should a plugin preferably use check_cluster_access() and get_connected_manager() instead of accessing cluster_managers directly? Likewise, should PBS operations call check_pbs_access() before resolving pbs_managers[pbs_id]?

2. Methods such as _api_get, _api_post, _wait_for_task and the existing SSH helpers currently have private-style names. Should they be treated as supported plugin integration seams, or should community plugins expect them to change and feature-detect them through a compatibility layer?

3. Which existing permission should gate destructive PBS LUN operations such as formatting or signature removal: pbs.config, pbs.datastore.create, or another permission? I could not find a dedicated permission such as pbs.disks.format, and I would prefer not to invent one that PegaProx cannot manage.

For now, I will keep the plugin UI global and treat PBS-specific navigation/context integration as non-blocking unless there is already a preferred hook.

The read-only discovery milestone will not perform iSCSI login, mounting, formatting, signature removal or datastore registration. Destructive execution will remain deferred until identity verification, explicit confirmation, durable operation state, cancellation checkpoints and complete audit coverage are implemented.

Thanks again — this already removes several important architectural unknowns and should prevent substantial rework.

[mkellermann97]

Great questions — all three have concrete answers.

1. Access helpers vs. raw dicts — yes, use the helpers. For PVE the order is authz first, then resolve: check_cluster_access(cluster_id) (in pegaprox.api.helpers) returns (True, None) or (False, <403 response>), and get_connected_manager(cluster_id) returns (manager, None) or (None, <404/503 response>). So gate on check_cluster_access before you touch the manager, then resolve it via get_connected_manager rather than indexing cluster_managers yourself — that way you inherit the tenant / VM-ACL / pool checks and the not-connected handling for free. Same on the PBS side: call check_pbs_access(pbs_id) before resolving pbs_managers[pbs_id]. Reaching into the global dicts directly is fine only for un-authz'd internal work; anything reachable from a route should go through the checks.

2. The _api_* / SSH helpers — treat them as internal; adapter + feature-detect is the right call. They're what core uses and they've been stable in practice, but they're deliberately underscore-private and we don't formally freeze them as a plugin API yet — so your instinct to keep them behind a thin compatibility adapter and feature-detect is exactly right, and it's what I'd do. If you end up leaning hard on one specific seam, raise it and we can look at promoting that one to a documented, stable interface rather than you having to guess.

3. Destructive PBS disk perms — there isn't one yet, and you shouldn't invent it. The catalog (pegaprox/models/permissions.py) is static and core-owned — a plugin can't register a slug the RBAC UI will actually grant. Today the only disk perms are read-only (pbs.disks.view, pbs.disks.smart); there's no writable pbs.disks.*. For the read-only discovery milestone, gate on those plus pbs.datastore.view / pbs.view. For the later destructive milestone, don't overload pbs.config or pbs.datastore.create — the semantics are wrong. Ping me when you get there and I'll add a dedicated pbs.disks.manage (format / signature-removal) perm to the catalog so it's properly grantable — that's a one-line core change and the correct home for it.

Keeping the destructive layer deferred behind exactly that kind of explicit perm + audit line is the right shape. Good stuff.

[mdobprv-lab]

Thank you again for the concrete guidance. I have now implemented it in the 0.2.0 read-only discovery preview:

• Release: https://github.com/mdobprv-lab/pegaprox-storage-assistant/releases/tag/v0.2.0
• Implementation PR: feat: add read-only PVE/NFS and PBS/iSCSI discovery mdobprv-lab/pegaprox-storage-assistant#3

The read-only milestone now:

• calls check_cluster_access() before resolving a PVE manager;
• calls check_pbs_access() before reading pbs_managers[pbs_id];
• reuses PegaProx-managed PVE/PBS connections without copying credentials;
• isolates feature-detected private API and SSH seams in compat.py;
• requires storage.view for PVE inspection and pbs.view, pbs.disks.view and pbs.datastore.view for PBS inspection;
• intentionally does not request pbs.disks.smart, because no SMART data is read;
• runs bounded daemon workers with owner-scoped results and cooperative cancellation;
• publishes minimal PVE progress through cluster-scoped broadcast_sse;
• retains owner-scoped REST polling for PBS rather than emitting an unscoped/global SSE event;
• audits discovery start, cancellation request and final state.

The execution boundary remains deliberately strict. Version 0.2.0 does not run SendTargets discovery, log in to iSCSI targets, mount storage, format filesystems, remove signatures or register datastores. Resource values are not interpolated into remote commands, and optional PBS SSH inspection is restricted to a hard-coded read-only allowlist with time and output bounds.

The implementation passed 70 automated tests and was exercised in a healthy PegaProx 1.0.2 Docker deployment. The runtime registry remained unchanged across installation, restart and discovery/cancellation tests.

We will keep mutating PBS work blocked until the core-owned pbs.disks.manage permission and suitable public integration seams are available. If you notice any remaining architectural mismatch in this read-only milestone, an early warning would still be very valuable; otherwise, we will continue from this boundary rather than widening the plugin prematurely.