✨ Add an optional packaged QDMI Driver extension

.agent/plans/qdmi-default-driver-extension-c2.md

@@ -0,0 +1,52 @@
+# Optional packaged QDMI driver extension
+
+Status: independently rebased and validated locally.
+
+## Motivation and settled boundary
+
+Installed providers need catalogue discovery and targeted stable-ID opening
+without importing vendor Python code or copying libraries beside the driver.
+Core's packaged driver provides an optional private extension for this purpose.
+The standard Client ABI remains usable with drivers that lack that extension.
+Standardizing discovery/configuration is QDMI v2 work.
+
+This is Core #2230 on #2229, targeting Core 4.1 / QDMI 1.4. It does not depend
+on metadata removal, batching, payload capabilities, or compiler changes.
+
+## Implementation
+
+- Load the two optional private symbols in the Client wrapper. Staging a
+  manifest does not select the process-wide driver; successful raw targeted
+  allocation does, even when subsequent initialization fails.
+- Stage trusted manifests transactionally at lowest precedence, freeze the
+  registry after successful construction, and keep canonical paths idempotent.
+- Open exactly one stable ID with strict per-call overrides. Preserve sized
+  custom values, reject malformed paths/IDs, and propagate provider errors.
+- Discover Python manifests using entry-point metadata and wheel RECORD paths,
+  never provider imports. Invalid automatic entries warn and are skipped;
+  explicit staging remains strict.
+- Retain initialized provider libraries across independent sessions. Complete
+  initialization before moving the session owner, including on Windows.
+
+The entry points are qdmi::default_driver::addManifest/openDevice and the Python
+default_driver submodule. No public QDMI C header changes are needed.
+Installed-consumer packaging is the separate follow-up #2231.
+
+## Acceptance
+
+Run the independent release build and CTest suite, generated stubs, QDMI/SDK
+Python tests, repository lint, and C++ lint. Check absent optional symbols,
+selection timing, freeze rollback, idempotent paths, strict overrides, valid
+warning outputs, malformed JSON, UTF-8 paths, and session lifetime. Discovery
+tests must reject missing RECORD, ambiguous/off-anchor paths and traversal while
+proving that provider code is not imported. Retain current optional-device
+configurations, concurrency, and compiler rules.
+
+The release suite passed 3,873 tests with one existing skip. All 467 selected
+Python tests passed. Stub generation, repository lint and C++ lint passed.
+
+## Recovery and non-goals
+
+Keep useful commits, human attribution and review threads. Use guarded pushes;
+do not create archive branches or request reviews. There is no new registry API,
+public discovery standard, scheduler policy, or payload contract here.

CHANGELOG.md

@@ -106,6 +106,9 @@ releases may include breaking changes.
 - 💥 Replace the QDMI-specific primitives with native Qiskit primitives and
   typed backend factories. Sampler and `memory=True` require genuine QDMI
   `SHOTS` ([#2358]) ([**@burgholzer**])
+- ✨ Discover installed device catalogues without importing provider modules and
+  open targeted sessions through the packaged driver's optional private
+  configuration extension ([#2230]) ([**@burgholzer**])
 - 💥 Load one replaceable QDMI 1.4 Client driver through a validated function
   table, split `MQT::CoreQDMI` from the packaged `MQT::CoreQDMIDriver`, and use
   stable Client device IDs across C++, Python, MLIR, Qiskit, PennyLane, and
@@ -911,6 +914,7 @@ for previous changelogs._
 [#2257]: https://github.com/munich-quantum-toolkit/core/pull/2257
 [#2246]: https://github.com/munich-quantum-toolkit/core/pull/2246
 [#2240]: https://github.com/munich-quantum-toolkit/core/pull/2240
+[#2230]: https://github.com/munich-quantum-toolkit/core/pull/2230
 [#2229]: https://github.com/munich-quantum-toolkit/core/pull/2229
 [#2232]: https://github.com/munich-quantum-toolkit/core/pull/2232
 [#2228]: https://github.com/munich-quantum-toolkit/core/pull/2228

UPGRADING.md

@@ -31,8 +31,17 @@ The generic session parameters are `token`, `auth_file`, `auth_url`, `username`,
 `password`, `project_id`, and `custom1` through `custom5`. The selected Driver
 owns their validation and meaning. The former Python `base_url`,
 `device_config`, and `device_config_file` keywords were tied to MQT Core's
-Driver and are not part of the generic Client boundary. Use persistent Driver
-configuration when that Driver supports these settings.
+Driver and are not part of the generic Client boundary. Code that deliberately
+requires MQT Core's packaged Driver can use
+`mqt.core.qdmi.default_driver.open_device` for these settings. Otherwise, use
+persistent Driver configuration.
+
+The `mqt.core.qdmi.default_driver` submodule is an optional MQT Core extension,
+not a standard QDMI Client API. Its `add_manifest` function stages trusted
+package manifests, and its `open_device` function opens one configured stable ID
+without enumerating other devices. Generic `ClientSession` and `open_device`
+calls never require or use this extension. Third-party Client drivers may omit
+it.
 
 The MLIR `from_device_id` helpers and the Qiskit and PennyLane adapters accept
 the same generic session parameters. Device, site, operation, and job wrappers

bindings/qdmi/qdmi.cpp

@@ -9,6 +9,7 @@
  */
 
 #include "qdmi/Client.hpp"
+#include "qdmi/common/Common.hpp"
 
 #include <nanobind/nanobind.h>
 #include <nanobind/operators.h>
@@ -66,6 +67,61 @@ qdmi::SessionConfig makeClientSessionConfig(
   };
 }
 
+[[nodiscard]] auto makeDeviceSessionJson(
+    const std::optional<std::string>& baseUrl,
+    const std::optional<std::string>& token,
+    const std::optional<std::filesystem::path>& authFile,
+    const std::optional<std::string>& authUrl,
+    const std::optional<std::string>& username,
+    const std::optional<std::string>& password,
+    const std::optional<std::string>& deviceConfig,
+    const std::optional<std::filesystem::path>& deviceConfigFile,
+    const std::optional<std::string>& custom1,
+    const std::optional<std::string>& custom2,
+    const std::optional<std::string>& custom3,
+    const std::optional<std::string>& custom4,
+    const std::optional<std::string>& custom5) -> std::string {
+  if (deviceConfig && deviceConfigFile) {
+    throw nb::value_error(
+        "device_config and device_config_file are mutually exclusive");
+  }
+  nb::dict session;
+  const auto setString = [&session](const char* key,
+                                    const std::optional<std::string>& value) {
+    if (value) {
+      session[key] = *value;
+    }
+  };
+  setString("base-url", baseUrl);
+  setString("token", token);
+  if (authFile) {
+    session["auth-file"] = qdmi::detail::pathToUtf8(*authFile);
+  }
+  setString("auth-url", authUrl);
+  setString("username", username);
+  setString("password", password);
+  setString("custom1", custom1);
+  setString("custom2", custom2);
+  setString("custom3", custom3);
+  setString("custom4", custom4);
+  setString("custom5", custom5);
+  if (deviceConfig) {
+    nb::dict source;
+    source["inline"] =
+        nb::module_::import_("json").attr("loads")(*deviceConfig);
+    session["device-config"] = std::move(source);
+  } else if (deviceConfigFile) {
+    nb::dict source;
+    source["file"] = qdmi::detail::pathToUtf8(*deviceConfigFile);
+    session["device-config"] = std::move(source);
+  }
+  if (session.empty()) {
+    return {};
+  }
+  return nb::cast<std::string>(
+      nb::module_::import_("json").attr("dumps")(session));
+}
+
 template <typename Query>
 [[nodiscard]] nb::object queryCustomValue(Query query,
                                           const nb::handle valueType) {
@@ -105,7 +161,9 @@ template <typename Query>
 } // namespace
 
 NB_MODULE(MQT_CORE_MODULE_NAME, qdmiModule) {
-  qdmiModule.doc() = "QDMI Client entities.";
+  qdmiModule.doc() = "QDMI Client entities and MQT Core's default driver.";
+  auto defaultDriver = qdmiModule.def_submodule(
+      "default_driver", "Configure MQT Core's packaged QDMI Client driver.");
   bindings::registerSlurm(qdmiModule);
 
   nb::class_<qdmi::Session>(qdmiModule, "ClientSession",
@@ -707,6 +765,48 @@ when the custom slot is unsupported.)pb");
                 nb::sig("def __eq__(self, arg: object, /) -> bool"));
   operation.def(nb::self != nb::self,
                 nb::sig("def __ne__(self, arg: object, /) -> bool"));
+
+  defaultDriver.def("add_manifest", &qdmi::default_driver::addManifest,
+                    "manifest_path"_a,
+                    "Stage one installed package manifest before the default "
+                    "driver freezes.");
+
+  defaultDriver.def(
+      "open_device",
+      [](const std::string& deviceId,
+         const std::optional<std::filesystem::path>& driverPath,
+         const std::optional<std::string>& baseUrl,
+         const std::optional<std::string>& token,
+         const std::optional<std::filesystem::path>& authFile,
+         const std::optional<std::string>& authUrl,
+         const std::optional<std::string>& username,
+         const std::optional<std::string>& password,
+         const std::optional<std::string>& deviceConfig,
+         const std::optional<std::filesystem::path>& deviceConfigFile,
+         const std::optional<std::string>& custom1,
+         const std::optional<std::string>& custom2,
+         const std::optional<std::string>& custom3,
+         const std::optional<std::string>& custom4,
+         const std::optional<std::string>& custom5) {
+        return qdmi::default_driver::openDevice(
+            deviceId,
+            makeDeviceSessionJson(baseUrl, token, authFile, authUrl, username,
+                                  password, deviceConfig, deviceConfigFile,
+                                  custom1, custom2, custom3, custom4, custom5),
+            driverPath);
+      },
+      "device_id"_a, nb::kw_only(), "driver_path"_a = std::nullopt,
+      "base_url"_a = std::nullopt, "token"_a = std::nullopt,
+      "auth_file"_a = std::nullopt, "auth_url"_a = std::nullopt,
+      "username"_a = std::nullopt, "password"_a = std::nullopt,
+      "device_config"_a = std::nullopt, "device_config_file"_a = std::nullopt,
+      "custom1"_a = std::nullopt, "custom2"_a = std::nullopt,
+      "custom3"_a = std::nullopt, "custom4"_a = std::nullopt,
+      "custom5"_a = std::nullopt,
+      "Open one device through MQT Core's strict private driver extension.");
+
+  nb::module_::import_("mqt.core._qdmi_discovery")
+      .attr("discover_qdmi_manifests")(defaultDriver.attr("add_manifest"));
 }
 
 } // namespace mqt

docs/qdmi/configuration.md

@@ -81,7 +81,8 @@ included in Driver warnings.
 
 Definitions are merged field by field by ID, from lowest to highest precedence:
 
-1. generated `*.qdmi.json` fragments packaged beside the MQT Core Driver;
+1. generated `*.qdmi.json` fragments packaged beside the MQT Core Driver and
+   trusted manifests staged by installed packages;
 2. the system `qdmi.json`;
 3. the user or XDG `qdmi.json`;
 4. the nearest project `qdmi.json`;
@@ -101,6 +102,50 @@ a device that an administrator disabled.
 `MQT_CORE_QDMI_CONFIG_FILE` replaces the system, user, and project levels while
 retaining packaged built-ins.
 
+## Installed Python package manifests
+
+A Python distribution can advertise one trusted device manifest without
+importing its provider package. Add an entry point to the distribution's
+`pyproject.toml`:
+
+```toml
+[project.entry-points."mqt.core.qdmi.manifests"]
+"example.qdmi.json" = "vendor.device"
+```
+
+The entry-point name must be the exact, path-free basename of one `*.qdmi.json`
+file. The value must be a dotted Python module name that anchors the owning
+package. The distribution's wheel `RECORD` must contain exactly one file with
+that basename below the corresponding module path. In this example, the path
+starts with `vendor/device/`. MQT Core resolves the file through the
+distribution metadata. It does not load the entry point or import the provider
+module.
+
+Importing {py:mod}`mqt.core.qdmi` stages every valid advertised manifest in the
+packaged Driver's lowest-precedence layer. Invalid or ambiguous automatic
+entries cause one `RuntimeWarning` each and are skipped. A metadata enumeration
+failure causes one warning and skips automatic discovery. Applications can stage
+a known manifest explicitly when an error must stop startup:
+
+```python
+from pathlib import Path
+
+from mqt.core.qdmi import default_driver
+
+default_driver.add_manifest(Path("vendor/device/example.qdmi.json"))
+```
+
+Explicit staging reports malformed manifests, missing libraries, and conflicting
+device IDs as errors. Staging the same canonical path more than once is
+idempotent, including after the packaged Driver freezes its registry. A new path
+cannot be staged after the packaged Driver successfully constructs and freezes
+its registry during a session-allocation request. A failed Driver construction
+rolls the freeze back so startup can be retried. Staging loads the packaged
+Driver library but does not select it as the process's generic Client driver.
+Package staging and default targeted opens ignore `MQT_CORE_QDMI_DRIVER` and use
+the packaged Driver. An explicit targeted `driver_path` overrides that default.
+Standard Client sessions continue to honor the environment override.
+
 ## Using configured devices
 
 When the packaged QDMI Driver initializes a Client session, it opens the
@@ -180,9 +225,9 @@ Built-in targets generate manifests beside their runtime libraries in both build
 and install trees. Library paths in those fragments contain only the target
 filename, so moving an installed tree or Python wheel preserves discovery.
 Automatic discovery searches relative to the MQT Core Driver, not every library
-loaded by the process. An application using a separately installed device
-implementation therefore copies its manifest beside the Driver or registers its
-definition by stable ID.
+loaded by the process. A separately installed Python distribution can use the
+`mqt.core.qdmi.manifests` entry point described above. Other applications copy
+the manifest beside the Driver or register the definition by stable ID.
 
 A fully static executable has no portable shared-module location. Place the
 fragments beside the executable, point `MQT_CORE_QDMI_CONFIG_FILE` at a complete

docs/qdmi/driver.md

@@ -36,6 +36,45 @@ different Driver fails. A failed load, ABI check, symbol check, or raw-session
 allocation does not select a Driver, so a later call can retry. MQT Core keeps
 the selected shared library loaded while its function pointers can be used.
 
+## Optional Packaged-Driver Extension
+
+MQT Core's packaged Driver adds two private symbols to the same shared library
+that exports the standard Client interface:
+
+- `MQT_CORE_QDMI_driver_add_manifest_v1` stages a trusted package manifest.
+- `MQT_CORE_QDMI_driver_session_alloc_for_device_v1` allocates a session for one
+  configured stable ID.
+
+These symbols are an MQT Core extension. They are not part of a public QDMI
+header, and another Client driver can omit them. MQT Core resolves them as
+optional symbols and calls them only through the extension API. Missing
+extension symbols do not prevent standard Client sessions. Generic
+{cpp-api:func}`qdmi::Session::openDevice` and Python
+{py:func}`mqt.core.qdmi.open_device` enumerate the standard Client device list
+and never use the private targeted-session symbol.
+
+Use {cpp-api:func}`qdmi::default_driver::addManifest` or Python
+{py:func}`mqt.core.qdmi.default_driver.add_manifest` before the packaged Driver
+freezes its registry. Staging the packaged library does not select it as the
+generic Client driver. The first successful raw standard or targeted session
+allocation selects a Client driver. A later targeted-session initialization or
+device query failure does not undo that selection.
+
+By default, the `default_driver` extension resolves MQT Core's packaged Driver
+and ignores `MQT_CORE_QDMI_DRIVER`. An explicit `driver_path` overrides that
+default for a compatible extension. The process selection rule still prevents
+switching Drivers after a successful raw allocation. Standard Client sessions
+use the selection order above, including the environment override.
+
+Use {cpp-api:func}`qdmi::default_driver::openDevice` or Python
+{py:func}`mqt.core.qdmi.default_driver.open_device` when an application
+deliberately depends on the packaged Driver. The targeted call merges manifest
+defaults with the supplied JSON or Python overrides. It rejects unsupported
+parameters and malformed configuration, propagates device-library status codes,
+and requires the session to expose exactly one device. Each call creates an
+independent session. The returned device and its derived wrappers retain that
+session until the last wrapper is destroyed.
+
 ## Building the Bundled Devices
 
 Standalone MQT Core builds include the DDSIM and superconducting QDMI device

include/mqt-core/qdmi/Client.hpp

@@ -494,6 +494,20 @@ class Site;
 class Device;
 class Operation;
 
+namespace default_driver {
+/// Stage one package manifest in MQT Core's optional driver extension.
+void addManifest(const std::filesystem::path& path);
+
+/// Open one default-driver device with strict merged session configuration.
+/// @param id Stable device ID.
+/// @param deviceSessionJson JSON session overrides.
+/// @param driverPath Optional compatible extension path. By default, this call
+/// uses MQT Core's packaged Driver and ignores `MQT_CORE_QDMI_DRIVER`.
+[[nodiscard]] Device openDevice(
+    std::string_view id, std::string_view deviceSessionJson = {},
+    const std::optional<std::filesystem::path>& driverPath = std::nullopt);
+} // namespace default_driver
+
 /**
  * @brief Class representing the Session library.
  * @details This class provides methods to query available devices and
@@ -804,6 +818,10 @@ class Device {
   Device(QDMI_Device device, std::shared_ptr<detail::ClientSession> session)
       : device_(device), session_(std::move(session)) {}
 
+  friend Device
+  default_driver::openDevice(std::string_view, std::string_view,
+                             const std::optional<std::filesystem::path>&);
+
   /// Wrap operation handles while retaining their owning device session.
   [[nodiscard]] std::vector<Operation>
   wrapOperations(std::span<const QDMI_Operation> operations) const;