diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml new file mode 100644 index 0000000..8ea0878 --- /dev/null +++ b/.github/workflows/docs.yml @@ -0,0 +1,47 @@ +name: docs + +on: + pull_request: + paths: + - docs/** + - mkdocs.yml + push: + branches: + - master + paths: + - docs/** + - mkdocs.yml + +permissions: + contents: read + +concurrency: + group: pages + cancel-in-progress: false + +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: astral-sh/setup-uv@v4 + - name: Build docs + run: uv run --with mkdocs-material==9.7.6 mkdocs build --strict + - uses: actions/upload-pages-artifact@v3 + with: + path: site + + deploy: + if: github.event_name == 'push' && github.ref == 'refs/heads/master' + needs: build + runs-on: ubuntu-latest + permissions: + pages: write + id-token: write + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + steps: + - uses: actions/configure-pages@v5 + - id: deployment + uses: actions/deploy-pages@v4 diff --git a/.gitignore b/.gitignore index 49b36de..631ea3d 100644 --- a/.gitignore +++ b/.gitignore @@ -23,4 +23,7 @@ coverage.html *.swp # Claude -.claude/scheduled_tasks.lock \ No newline at end of file +.claude/scheduled_tasks.lock + +# Docs site build output +site/ \ No newline at end of file diff --git a/docs/concepts/architecture.md b/docs/concepts/architecture.md new file mode 100644 index 0000000..58c3d7a --- /dev/null +++ b/docs/concepts/architecture.md @@ -0,0 +1,215 @@ +# Architecture + +YACD is a Kubernetes-native development environment manager for Cardano. It is +built for people developing applications on Cardano, not for validators, stake +pool operators, or production network operators. This page explains how the +system is put together and why it is shaped the way it is. For step-by-step +instructions see the [developer](../developer/getting-started.md) and +[operator](../operator/installation.md) guides; for exhaustive field and flag +facts see the [reference](../reference/cardanonetwork.md) section. + +## Two artifacts: operator and CLI + +YACD ships as two cooperating pieces: a Kubernetes operator and a companion +`yacd` CLI. The split is deliberate, and it follows the grain of Kubernetes +itself. + +The operator owns **declarative cluster state**. It reconciles long-lived +desired state: node workloads, generated or fetched configuration, genesis +material, supporting services, persistent volumes, Secrets, ConfigMaps, +Services, and the status that reports whether all of that is healthy. If you can +sensibly express it as "this environment should exist, with these services +enabled," it belongs to the operator. The operator watches its custom resources +across namespaces and continuously drives the cluster toward the spec. + +The CLI owns **imperative developer workflow**. It compiles a single +developer-facing config file into the Kubernetes resources the operator +consumes, applies them, waits for readiness, prints connection details, +forwards endpoints to your host, and runs one-off actions like topping up a +wallet. These are ad hoc commands, not desired state. + +The reason for the boundary is that Kubernetes is an excellent model for desired +state and a poor model for one-off actions. "This network should run with +db-sync enabled" is a natural fit for a controller that reconciles toward it +forever. "Top this address up right now" is not: encoding a single funding +request as a resource mutation would be awkward and would leave stale objects +behind. So durable state lives in CRDs and the operator, and transient actions +live in the CLI. The CLI is a Kubernetes client; it holds no privileged state of +its own and talks to the same API server any other client would. + +## CardanoNetwork: the primary resource + +The primary custom resource is `CardanoNetwork` (API group `yacd.meigma.io`, +version `v1alpha1`). It is intentionally environment-shaped rather than +node-shaped: it describes a Cardano *network* and the chain-access services +exposed beside it, even though the first runtime reconciles a single primary +node. + +A `CardanoNetwork` owns the core Cardano substrate. Its spec carries the network +`mode` (`local` or `public`), the primary `node` workload (shared by both +modes), the mode-specific `local` or `public` block, and a `chainAPI` block for +the services exposed next to the node. The controller reconciles the node into a +workload backed by a PVC for the node database, publishes resolved network +identity and cluster-local Service endpoints into status, and tracks health +through a set of `metav1.Condition` entries (`Ready`, `NodeReady`, +`NodeSynchronized`, `OgmiosReady`, `KupoReady`, `ArtifactsReady`, and others). +Status reports only what the controller can +observe in-cluster, so consumers do not trust stale or hand-edited values. + +Two chain-access services are enabled by default, because a raw `cardano-node` +is not enough to build against. [Ogmios](https://ogmios.dev) is the default +chain-access API: it gives YACD and developers a JSON/RPC interface for query, +submit, and evaluate without every client having to share the node's Unix +socket. [Kupo](https://cardanosolutions.github.io/kupo/) is the default chain +index. Funding is CLI-native: every local network is created with a +genesis-funded `faucet` wallet, and the `yacd wallet` verbs build and submit +funding transactions directly over Ogmios and Kupo — there is no in-cluster +faucet service. For the exact defaults, ports, and images, see the +[CardanoNetwork reference](../reference/cardanonetwork.md). + +## Supporting-service CRDs: CardanoDBSync + +Heavier services are modeled as **separate CRDs with separate controllers**, +rather than as ever-growing fields on `CardanoNetwork`. The first such service +is `CardanoDBSync`, which runs +[cardano-db-sync](https://github.com/IntersectMBO/cardano-db-sync) and its +Postgres database. + +A supporting CRD references a same-namespace `CardanoNetwork` through its +`networkRef` and derives chain information from that network rather than +re-declaring it. This keeps each controller focused: the db-sync controller +reasons about db-sync, Postgres, and ledger state, while the network controller +reasons about the Cardano substrate. It also keeps ownership clean. Enabling +db-sync does not have to disturb the primary node, and the db-sync controller +owns its own workload, storage, database wiring, config, and status. + +The default placement is a **dedicated follower node**. Rather than forcing the +primary node to share its Unix socket, the db-sync controller runs its own +follower `cardano-node` colocated with db-sync in one workload. The follower +joins the primary network over ordinary node-to-node TCP and exposes a local +socket that db-sync consumes inside the same Pod. This costs extra CPU, memory, +storage, and startup time, but it preserves controller isolation: the primary +node never restarts because you added an indexer. + +A `primarySidecar` placement mode is the explicit exception. When the cost of a +duplicate node outweighs the benefit of isolation, db-sync can be composed +directly into the primary node Pod and consume the primary socket. That trades +isolation for a single node copy and rolls the primary workload when the +attachment changes; it is supported for local and non-mainnet public profiles, +but not for public mainnet. For the field-level details of both modes, see the +[CardanoDBSync reference](../reference/cardanodbsync.md) and the +[db-sync operator guide](../operator/db-sync.md). + +## Why a Unix socket forces this shape + +Many Cardano tools want direct access to a node's Unix socket, and that single +fact drives much of the architecture. A Unix socket is a local-filesystem IPC +object. It is not a cluster-wide endpoint and cannot be exposed through a +Kubernetes Service. The robust pattern is to share it *within one Pod* using an +ephemeral volume: the node data directory lives on a PVC, while the socket +directory is ephemeral and mounted by sidecars in the same Pod. + +This is why Ogmios runs as a sidecar that mounts the node socket and re-exposes +chain access as a network API, and why socket-hungry services like db-sync +default to a colocated follower node instead of reaching across Pods. YACD +deliberately avoids RWX PVCs and hostPath socket sharing, which are fragile, +scheduler-sensitive, and a poor fit for shared clusters. The same constraint +shows up at the CLI boundary: tools that speak a network protocol can be +port-forwarded to your host, but a tool that needs the node socket directly +(notably `cardano-cli`) must run *inside* the node Pod. + +## Local vs public networks + +The two network modes differ mostly in where chain material comes from. + +A **local** network is generated and owned by YACD. The controller produces +fresh genesis and node configuration in-cluster from the spec: network magic, +ledger era, slot and epoch timing, generated stake-pool topology, and a curated +genesis profile (for example a zero-fee preset for fast tests). The generated +artifacts are staged and served over HTTP from the node's PVC by a small +cardano-tools serve endpoint, alongside a `manifest.json`, so the node and any +follower nodes consume the same configuration from one source of truth. The +`ArtifactsReady` condition reports when that bundle is staged and being served. + +A **public** network joins a known profile: `preprod`, `preview`, or `mainnet`. +Instead of generating genesis, the controller fetches the published +configuration for the profile. For most profiles the node can sync from genesis, +but mainnet is far too large to sync that way in a development setting. For +mainnet, the spec requires a [Mithril](https://mithril.network) bootstrap: an +init container uses a Mithril client to download and verify a Cardano database +snapshot, seeding the node's PVC so the node starts from a recent point instead +of from genesis. The same HTTP artifact-serving pattern applies, so resolved +configuration is reachable in-cluster. + +!!! warning "Mainnet is gated" + A public mainnet `CardanoNetwork` requires a Mithril bootstrap, large + persistent storage, and an explicit opt-in (`--allow-mainnet` on the CLI). + It is not the default development path. See the + [CardanoNetwork reference](../reference/cardanonetwork.md) for the + bootstrap and storage fields. + +## Host access: bridging the cluster to your host + +The services a `CardanoNetwork` exposes are **cluster-internal** Service URLs. +That is correct for in-cluster consumers and for supporting controllers, but a +developer's tests and tools usually run on the host. The CLI's host-access verbs +bridge that gap. + +`run`, `connect`, and `exec` are the three ways across the boundary. `run` wraps +a single command (the primary CI path), exposing host-usable Ogmios/Kupo URLs +through a small `YACD_*` environment contract so tests read ordinary environment +variables instead of parsing a YACD file. `connect` holds supervised +port-forwards open in one terminal while you work in another, writing the URLs to +an `endpoints.json` file instead of wiring an environment. + +`run` and the wallet funding commands do not always port-forward. A +CardanoNetwork can advertise a directly reachable `externalURL` for Ogmios and +Kupo — `yacd devnet` pins one on `localhost` (a NodePort mapped to a host port), +and a platform team can front a shared cluster with an ingress and declare its +URL once. The CLI probes that URL and uses it when reachable, falling back to an +ephemeral port-forward otherwise; `connect` always forwards. Port-forwarding +stays the universal transport because it assumes nothing about how the cluster is +reached. + +`exec` is different, and it exists because of the socket constraint above. Tools +that speak a network protocol can be forwarded, but `cardano-cli` reaches the +node over its local Unix socket, which cannot be forwarded as a TCP port. So +`exec` runs the command *inside* the primary node Pod, where +`CARDANO_NODE_SOCKET_PATH` and the `YACD_*` variables are already set. The rule +is simple: forward network APIs to the host with `run`/`connect`; run +socket-bound tools in-pod with `exec`. For the verb behaviors, the `YACD_*` +table, and the `endpoints.json` schema, see the +[CLI reference](../reference/cli.md) and the +[connecting tools guide](../developer/connecting-tools.md). + +!!! note "Funding is CLI-native" + Local networks are created with a genesis-funded `faucet` wallet. The `yacd + wallet` verbs spend from it by building and signing transactions on the host + and submitting them over Ogmios and Kupo; wallet keys live in labeled + Kubernetes Secrets and never leave for a server-side signer. See the + [funding guide](../developer/funding.md) and the + [security model](security.md). + +## Developer and operator workflows + +The same system serves two overlapping audiences. + +A **developer** typically authors one developer-facing config file, checks it +into a repository, and drives it with the CLI: render and apply the network, +wait for readiness, fund a wallet, and point tests at the forwarded endpoints. +They rarely hand-author CRDs; the CLI compiles their single config into the +decomposed Kubernetes resources the operator expects, giving them one pane of +glass while preserving clean resource boundaries in the cluster. + +An **operator** installs and runs the YACD operator in a cluster (locally on +[k3d](https://k3d.io) or in a shared cluster), manages the manager Deployment +and its RBAC through the Helm chart, and reasons directly about CRDs, status +conditions, placement modes, and capacity for heavier services like db-sync. + +The two workflows overlap because they act on the same resources through the +same API server. A developer's `yacd up` produces the very `CardanoNetwork` an +operator can inspect with `kubectl`; an operator's installed controller is what +makes a developer's config become a running network. The CLI is a convenience +layer over the cluster API, not a separate control plane. For the operator's +trust boundaries and the host-access trust gates, see the +[security model](security.md). diff --git a/docs/concepts/security.md b/docs/concepts/security.md new file mode 100644 index 0000000..df0a470 --- /dev/null +++ b/docs/concepts/security.md @@ -0,0 +1,100 @@ +# Security model + +YACD is a development environment manager, and two facts shape its threat model: +wallet keys are real signing credentials, and the operator runs with cluster-wide +reach. This page explains the deliberate security decisions that follow: how +wallet keys are stored and signed with, what the cluster-scoped manager RBAC +means on a shared cluster, and how the chart's optional image-verification path +hardens the supply chain. For the concrete knobs, follow the links to the how-to +and reference pages. + +## Wallet key custody + +Funding a wallet means signing a real transaction, so wallet keys are credentials +worth protecting. YACD keeps custody simple and Kubernetes-native. + +- **Keys live in labeled Kubernetes Secrets.** Each managed wallet is a + `-wallet-` Secret in the network's namespace, holding the payment + signing key, verification key, and address. The genesis-funded `faucet` wallet + has the same shape (`-wallet-faucet`) and is created and owned by the + operator. Storing keys as Secrets means they inherit Kubernetes RBAC, + encryption-at-rest, and backup rather than living in a bespoke store. +- **The CLI signs locally; the cluster never signs for you.** `yacd wallet` reads + the source wallet's signing key, builds and signs the funding transaction on + your machine, and submits it over Ogmios. There is no server-side signing + endpoint and no faucet HTTP service to authenticate against, so no long-lived + spending credential is exposed inside the cluster. +- **`export` writes keys deliberately.** `yacd wallet export` is the only path + that puts raw key material on local disk; it writes `0600` files under a `0700` + directory and never prints keys to stdout. + +Because there is no in-cluster faucet service, there is no Bearer token to +distribute and no token-to-URL trust gate to reason about. Keys stay in Secrets, +and signing stays on the host. + +## Cluster-scoped manager RBAC + +The chart binds the manager's ServiceAccount to a `ClusterRole`, not a namespaced +`Role`. The grant is broad by design: the operator manages `CardanoNetwork` and +`CardanoDBSync` objects and reconciles them into the core Kubernetes workloads +they need. + +The `ClusterRole` grants, across all namespaces: + +- `configmaps` and `persistentvolumeclaims`: create, get, list, patch, update, + watch +- `pods`: get, list +- `secrets`: create, delete, get, list, patch, watch +- `services`: create, delete, get, list, patch, update, watch +- `deployments` (`apps`): create, get, list, patch, update, watch +- `cardanonetworks` and `cardanodbsyncs` (`yacd.meigma.io`): get, list, watch, + plus get, patch, update on their `/status` subresources + +The implication for a **shared cluster** is direct: a YACD install can create, +read, and delete Secrets, Services, ConfigMaps, PVCs, and Deployments in any +namespace, because that is the scope the reconcilers operate at. The `secrets` +verbs include `create` and `delete` because wallet keys and other runtime +material are managed Secrets. Treat the operator as a cluster-wide actor when you +decide where to install it; an isolated development cluster is the intended home, +not a multi-tenant production cluster shared with untrusted workloads. + +RBAC creation is on by default but optional (`rbac.create`), and the role and +binding names are templated so an operator can substitute externally managed +RBAC. See the [installation guide](../operator/installation.md) and the +[configuration reference](../reference/configuration.md) for those values. + +## Supply-chain image verification + +The release workflow attests the manager image and the Helm chart with +GitHub-native (Sigstore keyless) attestations. The chart ships an **opt-in** +Kyverno `ClusterPolicy` that verifies the manager image attestation at Pod +admission time, closing the gap between "an image is signed somewhere" and "only +verified images run in this cluster". + +The policy is **disabled by default** (`kyverno.imageVerification.enabled: +false`) because it depends on a running [Kyverno](https://kyverno.io) +installation. When you enable it, Kyverno intercepts pod admission and rejects +images that do not carry a valid keyless Sigstore attestation matching the +configured attestor. + +The attestor defaults encode trust in the `meigma/yacd` release pipeline rather +than in a long-lived key: + +- **issuer** `https://token.actions.githubusercontent.com` — the OIDC identity + must come from GitHub Actions. +- **subjectRegExp** restricts the signing workflow to `meigma/yacd`'s + `release.yml` running on a `v` tag, so a signature produced by any + other workflow or fork does not satisfy the policy. +- **Rekor** at `https://rekor.sigstore.dev` — the transparency log that records + the keyless signing event. +- **attestation** of type `https://slsa.dev/provenance/v1` with build type + `https://actions.github.io/buildtypes/workflow/v1` — the policy checks SLSA + provenance, not just a bare signature. + +Because the signing identity is keyless and tied to a specific workflow on a +release tag, there is no signing key to leak or rotate; trust flows from the +GitHub OIDC identity and the public transparency log. The issuer, subject +pattern, Rekor URL, validation failure action, and the set of image references +the policy applies to are all configurable under `kyverno.imageVerification.*`. +To enable and tune verification, see the [installation guide](../operator/installation.md); +for the full value surface, see the [configuration reference](../reference/configuration.md). diff --git a/docs/developer/connecting-tools.md b/docs/developer/connecting-tools.md new file mode 100644 index 0000000..4287021 --- /dev/null +++ b/docs/developer/connecting-tools.md @@ -0,0 +1,153 @@ +# Connecting tools & tests + +The operator publishes a network's chain APIs as **cluster-internal** +`*.svc.cluster.local` Service URLs that a laptop or CI runner cannot reach +directly. Use `yacd run`, `yacd exec`, and `yacd connect` to bridge that gap. +Each verb sets the same `YACD_*` environment variables, so your tools read +ordinary env vars and stay YACD-agnostic across local and CI runs. + +This page covers choosing and using the verbs. For the full variable list and +defaults, see the [CLI reference](../reference/cli.md). All examples assume the +network is already `up` and `Ready`. + +## Choose a verb + +Pick the verb by how your tool reaches the node and how long you need access: + +| You want to… | Use | Why | +|---|---|---| +| Run a test suite or one-off command against Ogmios or Kupo | [`yacd run`](#run-a-test-runner) | Resolves a host-usable URL per endpoint (a reachable `externalURL` or a loopback port-forward), injects `YACD_*`, runs your command, then tears any forwards down. | +| Run `cardano-cli` or any tool that needs the node's Unix socket | [`yacd exec`](#run-a-socket-bound-tool) | Runs the command inside the node Pod, where the socket is local. A port-forward cannot expose a Unix socket. | +| Keep endpoints open for a dev server, REPL, or repeated IDE test runs | [`yacd connect`](#hold-forwards-open) | Holds supervised forwards open in one terminal and writes the loopback URLs to `endpoints.json` for other processes to read. | + +The deciding factor between `run` and `exec` is the transport: **`run` reaches +the TCP chain APIs** (Ogmios and Kupo) from the host; **`exec` runs in the Pod** +for anything that speaks to the node over its local Unix socket. + +## Run a test runner + +Use `yacd run NAME -- ` for the primary test/CI path. It resolves a +host-usable URL for Ogmios and Kupo — preferring a reachable `externalURL` (for +example `yacd devnet`'s `localhost` ports) and falling back to a scoped +port-forward — injects the `YACD_*` environment, runs `` on the host, and +tears any forwards down when it exits. The command's exit code is propagated, so +a test failure survives the wrapper. The full resolution order is in the +[CLI reference](../reference/cli.md#chain-access-resolution). + +Put `--` before any command that takes its own flags so they pass through to the +command instead of being parsed by `yacd`: + +```sh +yacd run my-net -- go test ./e2e/... +``` + +Your test code reads the loopback URLs from the environment: + +```go +ogmios := os.Getenv("YACD_OGMIOS_URL") // reachable ws:// URL (externalURL or loopback forward) +kupo := os.Getenv("YACD_KUPO_URL") // reachable http:// URL +``` + +With no command, `run` drops into your `$SHELL` (falling back to `/bin/sh`) with +the same environment set, which is handy for poking at the network by hand: + +```sh +yacd run my-net +``` + +If a forward drops while the command is running, `run` cancels the command and +reports the lost connection instead of a bare exit code. + +See the [`YACD_*` variable table](../reference/cli.md) for every variable and +when it is present. + +## Run a socket-bound tool + +Use `yacd exec NAME -- ` for `cardano-cli` and anything that reaches the +node over its local Unix socket. `cardano-cli` talks to the node over a +`--socket-path`, not over TCP, so a port-forward cannot expose it. `exec` runs +the command **inside** the primary node Pod with `CARDANO_NODE_SOCKET_PATH` and +the `YACD_*` variables set in the pod environment, so `cardano-cli` finds the +socket automatically: + +```sh +yacd exec my-net -- cardano-cli query tip --testnet-magic 42 +``` + +`exec` runs the command directly, **not** through a shell, so `$VAR` references +in arguments are not expanded. To interpolate `YACD_*` variables into arguments, +run a shell explicitly: + +```sh +yacd exec my-net -- sh -c 'cardano-cli query tip --testnet-magic "$YACD_NETWORK_MAGIC"' +``` + +When both stdin and stdout are a terminal, `exec` attaches an interactive TTY +(raw mode, with window resizes forwarded), so this opens an interactive shell +inside the node Pod. Piped or non-terminal invocations (for example in CI) +stream without a TTY: + +```sh +yacd exec my-net -- sh +``` + +## Hold forwards open + +Use `yacd connect NAME` when several host processes need the endpoints at once, +or across repeated runs: a dApp dev server, a REPL, or an IDE that re-runs tests. +It holds supervised forwards open in the foreground (one terminal) and writes the +loopback URLs to a gitignored endpoint state file for other processes to read. +Run it in one terminal and your tools in another: + +```sh +yacd connect my-net +``` + +`connect` runs until you press Ctrl-C. If a forward drops (pod restart, idle +timeout) it re-resolves the primary Pod, reassigns local ports, and writes a +fresh endpoints file. A clean disconnect removes the file. + +The endpoint state file lives under `.yacd/`. When the namespace defaults to the +network name, the path is: + +```text +.yacd//endpoints.json +``` + +When `--namespace` is set, the path includes both identity parts so networks +with the same name in different namespaces do not collide: + +```text +.yacd///endpoints.json +``` + +Read the loopback URLs from that file in any host process. The file is created +`0600` under `0700` directories, holds no secrets, and its ports are only live +while `connect` is running. For the document's field names and shape, see the +[endpoints.json schema](../reference/cli.md). + +## Fund a wallet from a test + +`yacd wallet topup NET WALLET LOVELACE --await` funds a target and polls Kupo +until the funding transaction's output appears, so a test never races chain +inclusion. The `WALLET` argument is a managed wallet name or a bech32 +`addr_test...` address, and `topup` reaches Ogmios and Kupo on its own (a +reachable `externalURL` or a port-forward), so no `yacd run` wrapper is needed. +Pass `--ogmios-url` / `--kupo-url` to point it at specific endpoints: + +```sh +export ADDR=addr_test1... # the address your test funds +yacd wallet topup my-net "$ADDR" 1000000 --await +``` + +Funds come from the network's genesis `faucet` wallet by default; `--from` +selects another managed wallet. See the +[CLI reference](../reference/cli.md#wallet) for all `wallet` flags and the +[funding guide](funding.md) for the full wallet workflow. + +## See also + +- [CLI reference](../reference/cli.md) — every flag, the `YACD_*` variable + table, and the `endpoints.json` schema. +- [Architecture](../concepts/architecture.md) — why the contract is shaped this + way. diff --git a/docs/developer/funding.md b/docs/developer/funding.md new file mode 100644 index 0000000..3ed89d7 --- /dev/null +++ b/docs/developer/funding.md @@ -0,0 +1,68 @@ +# Fund an account on a local network + +Every local YACD network is created with a genesis-funded `faucet` wallet. You +spend from it to fund developer wallets (or any testnet address) with the `yacd +wallet` verbs, which build, sign, and submit transactions directly against the +network's Ogmios and Kupo endpoints. There is no in-cluster faucet service. + +!!! note "Local networks only" + The genesis-funded `faucet` wallet is created only for `local` networks; it + is funded at genesis from the localnet's initial UTxOs. Public networks + (preview/preprod/mainnet) have no faucet wallet — fund those from your own + funded keys. + +## Create and fund a wallet + +The quickest path is `yacd wallet add`, which generates a managed wallet and, +with `--topup`, funds it from the `faucet` wallet in one step. Add `--await` to +block until the funding transaction is confirmed on-chain: + +```sh +yacd wallet add my-net --topup 5000000 --await +``` + +This prints the new wallet's name and `addr_test...` address, the funding +transaction id, and `Confirmed on-chain.` once Kupo sees the output. Omit +`--topup` to create an unfunded wallet, and pass `--name` to choose the name +instead of the generated adjective-noun default. + +Managed wallet keys are stored as labeled Kubernetes Secrets +(`-wallet-`) in the network's namespace; the CLI reads them to +sign locally. See [Security](../concepts/security.md) for the custody model. + +## Fund an existing wallet or address + +`yacd wallet topup` funds an existing target with an exact lovelace amount. The +`WALLET` argument is a managed wallet name, a public key, or a bech32 +`addr_test...` address, so you can also fund an address you do not manage: + +```sh +yacd wallet topup my-net bright-sun 1000000 --await +yacd wallet topup my-net addr_test1... 1000000 +``` + +By default the funds come from the `faucet` wallet; pass `--from ` to +spend from another managed wallet instead. `topup` forwards Ogmios and Kupo +itself, so no `yacd run` wrapper or URL flags are needed. Add `--json` for a +machine-readable result. + +## List, export, and remove wallets + +```sh +yacd wallet list my-net +yacd wallet export my-net bright-sun +yacd wallet remove my-net bright-sun +``` + +`list` shows the wallets you created (name, address, and the `managed-by-cli` +source); the operator-owned `faucet` wallet is not listed. `export` writes the +wallet's `.skey`, `.vkey`, and +`.addr` files to `.yacd///wallets//` (override with +`--out`). The `faucet` wallet is reserved and operator-owned, so the CLI will not +remove it. + +## See also + +- [CLI reference](../reference/cli.md#wallet) — every `yacd wallet` flag and default. +- [Security](../concepts/security.md) — wallet key custody and local signing. +- [Connecting tools & tests](connecting-tools.md) — funding a wallet from a test run. diff --git a/docs/developer/getting-started.md b/docs/developer/getting-started.md new file mode 100644 index 0000000..9471679 --- /dev/null +++ b/docs/developer/getting-started.md @@ -0,0 +1,172 @@ +# Getting started + +This tutorial takes you from nothing to a running local Cardano devnet and a +funded address in a handful of commands. You will install the `yacd` CLI, bring +up a devnet, inspect it, query the chain tip, fund an address, and tear it all +down again. + +By the end you will have run a complete local development loop. For *why* the +pieces fit together this way, see +[Architecture](../concepts/architecture.md). + +!!! note "Prerequisites" + `yacd devnet` needs a running [Docker](https://www.docker.com/) (or a + compatible container runtime), because [k3d](https://k3d.io) runs the local + cluster as containers. You do not install k3d yourself: on first run the CLI + downloads a pinned, checksum-verified k3d binary and caches it under + `~/.local/share/yacd/bin`, so the first `yacd devnet` is slower than later + ones. + +## 1. Install the CLI + +Install the `yacd` binary using one of the options below. + +=== "Download a release" + + Download the binary for your platform from the project's GitHub Releases + page, make it executable, and move it onto your `PATH`: + + ```sh + chmod +x yacd + sudo mv yacd /usr/local/bin/yacd + ``` + +=== "Build from source" + + From a clone of the repository, build the CLI with the Go toolchain: + + ```sh + go build -o yacd ./cli/cmd/yacd + sudo mv yacd /usr/local/bin/yacd + ``` + +Verify the install: + +```sh +yacd --version +``` + +You should see a version line printed to your terminal. If the command is not +found, confirm the directory you moved `yacd` into is on your `PATH`. + +## 2. Bring up the devnet + +Create the cluster, install the operator, and apply a default funded network +with a single command: + +```sh +yacd devnet +``` + +This provisions a managed k3d cluster, installs the YACD operator into it, and +applies one network named `devnet` in the `devnet` namespace. Progress streams +to your terminal while it works, and on success it prints a summary like this: + +```text +devnet is ready. + Cluster: (context ) + Operator: + Ogmios: + Kupo: + Wallet: + +Try: + yacd exec devnet -- cardano-cli query tip --testnet-magic 42 + yacd devnet down +``` + +!!! note "Keep these handy" + The network uses network magic `42` (you will use it when you query the + chain), and the `Wallet` line shows the network's genesis-funded `faucet` + wallet, which funds the wallets you create. The two commands under `Try:` are + the next two things you will run. + +For what the cluster, operator, and network are and how they relate, see +[Architecture](../concepts/architecture.md). + +## 3. List and inspect the network + +List the networks in the cluster (`yacd list` shows every namespace by default): + +```sh +yacd list +``` + +The `devnet` network you just created appears in the output. Inspect it in +detail, including its status and connection information: + +```sh +yacd info devnet +``` + +`yacd info` prints the network's readiness and the endpoint URLs your tools +connect to. Keep this command handy while you work; it is the quickest way to +check what a network is exposing. + +!!! tip "devnet is reachable on localhost" + `yacd devnet` maps Ogmios and Kupo to fixed host ports, so you can point a + tool straight at `ws://localhost:1337` (Ogmios) and `http://localhost:1442` + (Kupo) — no port-forward. `yacd run` and the wallet commands use these + automatically. + +## 4. Query the chain tip + +Run `cardano-cli` *inside* the node Pod to query the current chain tip. This is +the first command from the `Try:` hint: + +```sh +yacd exec devnet -- cardano-cli query tip --testnet-magic 42 +``` + +`yacd exec` runs the command directly inside the primary node Pod, where the +node socket and the `YACD_*` environment are already set, so `cardano-cli` finds +the socket automatically. The `--testnet-magic 42` value is the devnet's network +magic from step 2. + +The output is a JSON object describing the tip (slot, block, epoch, and sync +percentage). Run it again after a few seconds and the slot advances, confirming +the chain is producing blocks. + +## 5. Fund a wallet + +Every local network is created with a genesis-funded `faucet` wallet. Create a +new managed wallet and fund it from the faucet in one command, waiting for +on-chain confirmation: + +```sh +yacd wallet add devnet --topup 5000000 --await +``` + +This generates a wallet, sends it 5,000,000 lovelace (5 ADA) from the `faucet` +wallet, and prints the new wallet's address and the funding transaction id. +`yacd wallet` builds and submits the transaction directly over Ogmios and Kupo; +there is no separate faucet service. + +!!! note "Wallets fund local development" + The `faucet` wallet and the wallets you create exist to fund development on + local networks. Public networks have no faucet wallet; fund those from your + own keys. + +For the full wallet workflow — funding an existing address, listing, exporting, +and removing wallets — see the [funding guide](funding.md). + +## 6. Tear it down + +When you are finished, delete the managed cluster: + +```sh +yacd devnet down +``` + +This removes the k3d cluster and everything in it. Your next `yacd devnet` will +start fresh. + +## Where to go next + +- [Working with networks](networks.md) — scaffold your own config with `yacd + init` and apply it. +- [Connecting tools](connecting-tools.md) — wire Ogmios, Kupo, and other tools + to a network from your host. +- [Funding](funding.md) — the full faucet and top-up workflow. +- [CLI reference](../reference/cli.md) — every command, flag, and default. +- [Architecture](../concepts/architecture.md) — why YACD is built this way. diff --git a/docs/developer/networks.md b/docs/developer/networks.md new file mode 100644 index 0000000..be7a0fc --- /dev/null +++ b/docs/developer/networks.md @@ -0,0 +1,161 @@ +# Defining networks + +This page collects task recipes for the network lifecycle: write an Environment +file, apply it, validate it without applying, inspect what is running, and tear +it down. Each `yacd` command keys off a `NAME` argument; the `NAME` becomes the +`CardanoNetwork` name and, unless you pass `--namespace`, the namespace as well, +so each environment lands in its own isolated namespace by default. + +!!! note "These commands need the operator" + `yacd up` and the verbs below assume the YACD operator is installed in the + target cluster. `yacd devnet` installs it into a local cluster for you; for + any other cluster, run [`yacd install`](../operator/installation.md) first. + +For what each Environment field means, see the +[Environment file reference](../reference/environment.md) and the +[CardanoNetwork reference](../reference/cardanonetwork.md). For complete, +copy-paste manifests, see [Recipes](../recipes.md). For every command flag and +default, see the [CLI reference](../reference/cli.md). + +## Scaffold and apply an Environment + +The quickest way to start is `yacd init`, which prints a fully-commented +Environment to stdout. Redirect it to a file: + +```sh +yacd init > yacd.yaml +``` + +The generated `yacd.yaml` is a ready-to-run local devnet: a single-pool network +with the faucet and a pre-funded wallet enabled. Its commented blocks document +the rest of the API — chain-API image and port overrides, and a public/mainnet +alternative — so you grow the config by uncommenting a whole block at a time. +Edit it to taste, then apply it: + +```sh +yacd up dev -f yacd.yaml +``` + +`yacd up` renders the Environment into a `CardanoNetwork` named `dev` in +namespace `dev`, creates the namespace if needed, server-side-applies the +network, and waits until it reports `Ready`. The identity (name and namespace) +comes from the command line, not the file. Use `--wait=false` to apply without +blocking, and `--timeout` to change the readiness deadline (default `12m`). See +the [CLI reference](../reference/cli.md) for the full flag set. + +!!! note + The `yacd init` template already spells these out, but if you write or trim a + file by hand: `spec.network.mode`, `spec.network.node.version`, and + `spec.network.node.port` must be written explicitly; the loader rejects a + document that omits them. Local mode additionally requires + `spec.network.local` (with `networkMagic`, `era`, `timing.slotLength`, + `timing.epochLength`, and `topology.pools.count`). See the + [Environment file reference](../reference/environment.md) for the required + fields per mode. + +## Validate without applying + +Use `--dry-run` to render the manifest and print it to stdout without touching +the cluster. This runs the same Environment validation and rendering as a real +apply, so it catches envelope, field, and mode errors: + +```sh +yacd up dev -f yacd.yaml --dry-run +``` + +The rendered `CardanoNetwork` is written to stdout; a `Dry run: rendered +CardanoNetwork dev/dev; no resources applied.` line is written to stderr. No +namespace is created and nothing is applied. + +## Inspect running environments + +List every `CardanoNetwork` across all namespaces: + +```sh +yacd list +``` + +The table shows `NAME`, `NAMESPACE`, `MODE`, `READY`, and a comma-separated +`ENDPOINTS` summary (for example `ogmios,kupo,faucet`, or `-` when nothing is +published yet). `READY` reflects a fresh `Ready` condition, so a stale status is +reported as not ready. + +Scope to a single namespace with `-n`, or emit machine-readable JSON with +`--json`: + +```sh +yacd list -n dev +yacd list --json +``` + +Print full status and connection details for one environment: + +```sh +yacd info dev +``` + +`yacd info` shows the network mode and magic, published endpoint URLs, faucet +and wallet status, and the `metav1.Condition` list. Add `--json` for a stable, +scriptable projection: + +```sh +yacd info dev --json +``` + +## Tear down + +Delete an environment and wait for it and its garbage-collected children to be +removed: + +```sh +yacd down dev +``` + +Deletion is idempotent: a network that is already absent is reported as success. +Use `--wait=false` to issue the delete without blocking, and `--timeout` to +change the removal deadline (default `5m`). + +## Run a public profile locally + +Public mode connects a node to a real Cardano network instead of a synthetic +local chain. The `preview` and `preprod` test networks need no bootstrap and are +the recommended way to develop against public-network data. + +To switch the scaffolded config to a public network, set `spec.network.mode: +public`, remove the `local:` block, and uncomment the `public:` block — the +`yacd init` template marks exactly what to change. Or save the preview manifest +from [Recipes](../recipes.md) as `yacd.yaml`. Then apply it the same way: + +```sh +yacd up preview -f yacd.yaml +``` + +Switch to preprod by setting `spec.network.public.profile: preprod`. For both +profiles, `spec.network.public.bootstrap` must be absent; it is accepted only +for `mainnet`. Public profiles sync from the network, so allow extra time and +storage compared with a local network. The copy-paste preview and preprod +manifests live in [Recipes](../recipes.md). + +## Run mainnet + +!!! warning + Mainnet is heavyweight and gated. A mainnet Environment **must** set + `spec.network.public.bootstrap.mithril` (the loader rejects a mainnet + document without it), provisions large persistent volumes, and bootstraps + chain state from [Mithril](https://mithril.network). `yacd up` refuses to + apply a mainnet network unless you pass `--allow-mainnet`; without it the + command fails with an explicit error. A `--dry-run` still renders a mainnet + manifest without the flag, but prints a warning and applies nothing. + +Save the mainnet manifest from [Recipes](../recipes.md) as `yacd.yaml`, then +apply it with the gate flag: + +```sh +yacd up mainnet -f yacd.yaml --allow-mainnet +``` + +Because the chain bootstrap and sync take far longer than a local or test +network, raise `--timeout` accordingly. See the +[CardanoNetwork reference](../reference/cardanonetwork.md) for the +`spec.public.bootstrap.mithril` fields and the +[Recipes](../recipes.md) page for the full mainnet manifest. diff --git a/docs/host-access.md b/docs/host-access.md deleted file mode 100644 index 8894362..0000000 --- a/docs/host-access.md +++ /dev/null @@ -1,147 +0,0 @@ -# Host access and the `YACD_*` contract - -The operator publishes a network's chain APIs as **cluster-internal** -`*.svc.cluster.local` Service URLs, which a laptop or CI runner cannot reach. -The `run`, `connect`, and `exec` verbs bridge that gap, and the `YACD_*` -environment variables are the **integration surface**: your tests read ordinary -env vars and never parse a YACD file, so the test runner stays YACD-agnostic and -the same test works locally and in CI. - -This page is a reference for that contract. It assumes a network is already -`up` and `Ready`. - -## The verbs - -| Verb | What it does | -|---|---| -| `yacd run NAME -- ` | Establish scoped port-forwards to the chain APIs, inject the `YACD_*` environment, run `` on the host, and tear the forwards down when it exits. No command drops into `$SHELL`. The command's exit code is propagated. This is the primary test/CI path. | -| `yacd connect NAME` | Hold the forwards open in the foreground (one terminal) while you work in another, writing the loopback URLs to `.yacd//endpoints.json`. Re-establishes dropped forwards; runs until Ctrl-C. | -| `yacd exec NAME -- ` | Run `` **inside** the primary node Pod, for tools that reach the node over its local Unix socket. | -| `yacd topup NAME LOVELACE --address ADDR` | Fund an address through the faucet, self-forwarding the faucet so no `yacd run` wrapper is needed. Add `--await` to wait for on-chain confirmation. | - -## `run` vs `exec`: which one? - -- Use **`run`** for tooling that speaks to **Ogmios, Kupo, or the faucet over - TCP** — it forwards those Services to loopback ports and points `YACD_*` at - them. -- Use **`exec`** for **`cardano-cli` and anything that needs the node's Unix - domain socket**. `cardano-cli` talks to the node over a local socket - (`--socket-path`), not TCP, so a port-forward cannot expose it; `exec` runs - the command in the Pod where the socket is local and sets - `CARDANO_NODE_SOCKET_PATH`. - -`exec` runs the command directly, **not** through a shell, so `$VAR` references -in arguments are not expanded. `cardano-cli` reads `CARDANO_NODE_SOCKET_PATH` -from the environment automatically: - -```sh -yacd exec my-net -- cardano-cli query tip --testnet-magic 42 -``` - -To interpolate `YACD_*` variables into arguments, run a shell explicitly: - -```sh -yacd exec my-net -- sh -c 'cardano-cli query tip --testnet-magic "$YACD_NETWORK_MAGIC"' -``` - -When stdin and stdout are a terminal, `exec` attaches an interactive TTY (raw -mode, with window resizes forwarded), so this opens an interactive shell inside -the node Pod; piped or non-terminal invocations (for example in CI) stream -without a TTY: - -```sh -yacd exec my-net -- sh -``` - -## The `YACD_*` environment variables - -The variable **names are identical** whether a command runs on the host -(`run`/`connect`) or inside the Pod (`exec`); only the values adapt, which is -what makes a test transparent to where it runs. This is contract **version 1**: -adding a variable is backward compatible; renaming or removing one is a breaking -change. - -| Variable | Host (`run`/`connect`) | In-pod (`exec`) | Present when | -|---|---|---|---| -| `YACD_NETWORK` | network name | same | always | -| `YACD_NAMESPACE` | namespace | same | always | -| `YACD_NETWORK_MAGIC` | e.g. `42` | same | the controller publishes a network magic | -| `YACD_OGMIOS_URL` | `ws://127.0.0.1:` | `ws://..svc.cluster.local:` | Ogmios published | -| `YACD_KUPO_URL` | `http://127.0.0.1:` | `http://..svc.cluster.local:` | Kupo published | -| `YACD_FAUCET_URL` | `http://127.0.0.1:` | `http://..svc.cluster.local:` | faucet published | -| `YACD_FAUCET_TOKEN` | faucet auth token | *(omitted)* | faucet ready, **host only** | -| `CARDANO_NODE_SOCKET_PATH` | — | `/ipc/node.socket` | `exec` only | - -The host URL schemes are taken from what the operator published (Ogmios stays -`ws://`), with the host rewritten to a random loopback port. `YACD_FAUCET_TOKEN` -is deliberately **never** set in the in-pod (`exec`) environment: a Bearer token -in the exec command line would leak into apiserver audit logs, and in-pod -tooling does not need it. - -## `connect` and endpoint state files - -`connect` writes the loopback URLs to a gitignored endpoint state file for -other host processes (a dApp dev server, a REPL, repeated IDE test runs) to -read. When `--namespace` is not set and the namespace defaults to the network -name, the path stays: - -```text -.yacd//endpoints.json -``` - -When `--namespace` is set, the path includes both identity parts so networks -with the same name in different namespaces do not collide: - -```text -.yacd///endpoints.json -``` - -The document shape is the same in either location: - -```json -{ - "network": "my-net", - "namespace": "my-net", - "networkMagic": 42, - "ogmiosUrl": "ws://127.0.0.1:34521", - "kupoUrl": "http://127.0.0.1:34522", - "faucetUrl": "http://127.0.0.1:34523" -} -``` - -The file is created `0600` under `0700` state directories and **never contains -the faucet token**. Its ports are only live while `connect` is running. A clean -disconnect removes the file, and a dropped forward removes the stale file before -re-establishing, reassigning local ports, and writing a fresh document. - -## Funding with `topup` - -`yacd topup NAME LOVELACE --address ADDR` funds `ADDR` through the faucet. -`topup` reaches the faucet on its own: with no `--faucet-url`, it opens a -short-lived port-forward to the faucet (and Kupo), POSTs, and tears it down — so -it works directly from the host with no `yacd run` wrapper: - -```sh -yacd topup my-net 1000000 --address "$ADDR" -``` - -Add `--await` to poll Kupo until the funding transaction's output appears, so a -test never races chain inclusion. When `topup` self-forwards it reuses that same -session's Kupo, so `--await` needs no extra flags: - -```sh -yacd topup my-net 1000000 --address "$ADDR" --await -``` - -`topup` also honors an ambient `YACD_FAUCET_URL`/`YACD_KUPO_URL`, so it still -works unchanged inside `yacd run` (no second forward is opened). The loopback -faucet URL — whether self-forwarded or inherited from `run`/`connect` — is exempt -from the `topup` trust gate, so no `--trust-faucet-url` flag is needed. An -explicit `--faucet-url` suppresses self-forwarding; a custom non-loopback value -then requires `--trust-faucet-url` (and `--allow-insecure-faucet-url` for -`http://`), and `--await` with an override needs an explicit `--kupo-url`. - -## See also - -- The full operator and CLI surface in the [README](../README.md). -- The architecture direction in [DESIGN.md](../DESIGN.md). diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..335bd16 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,56 @@ +# yacd + +yacd is a Kubernetes-native manager for Cardano development environments. It is +aimed at people building on [Cardano](https://developers.cardano.org), not +validators, stake pool operators, or production network operators. + +yacd ships as two surfaces: + +- A **Kubernetes operator** that owns declarative cluster state. The + `CardanoNetwork` CRD reconciles a Cardano node with [Ogmios](https://ogmios.dev) + as the default chain API, [Kupo](https://cardanosolutions.github.io/kupo/) as + the default chain index. Local networks also get a genesis-funded `faucet` + wallet for funding development. +- A companion **`yacd` CLI** that owns local developer workflow: standing up a + devnet, applying a checked-in config, waiting for readiness, printing + connection details, forwarding endpoints to your host, and funding wallets. + +## Where to go next + +=== "Developing locally" + + Build and test against a Cardano network on your own machine. Start with the + [Getting started tutorial](developer/getting-started.md). + +=== "Running on a cluster" + + Install the operator into an existing Kubernetes cluster — one `yacd install`, + or Helm — so a team can share YACD-managed networks. See + [Operator installation](operator/installation.md). + +## Try it in one command + +A single `yacd devnet` provisions a local cluster, installs the operator, and +applies a funded network. It needs [Docker](https://www.docker.com/) running and +fetches a pinned [k3d](https://k3d.io) binary on first use: + +```sh +yacd devnet +``` + +Follow the [Getting started tutorial](developer/getting-started.md) for the full +path, including how to inspect the network and wire it into your tests. Every +flag and default is listed in the [CLI reference](reference/cli.md). + +## External tools + +yacd composes existing Cardano and Kubernetes tooling. These links go to the +upstream projects: + +- [Cardano developer portal](https://developers.cardano.org) +- [Ogmios](https://ogmios.dev) — chain API +- [Kupo](https://cardanosolutions.github.io/kupo/) — chain index +- [cardano-db-sync](https://github.com/IntersectMBO/cardano-db-sync) — relational chain index +- [Mithril](https://mithril.network) — fast bootstrap for public networks +- [Helm](https://helm.sh) and [k3d](https://k3d.io) — packaging and the local cluster +- [Kubernetes](https://kubernetes.io) diff --git a/docs/operator/db-sync.md b/docs/operator/db-sync.md new file mode 100644 index 0000000..c2a1e6d --- /dev/null +++ b/docs/operator/db-sync.md @@ -0,0 +1,162 @@ +# DB-Sync indexing + +Index an existing `CardanoNetwork` with [cardano-db-sync](https://github.com/IntersectMBO/cardano-db-sync) +by applying a `CardanoDBSync` in the **same namespace**. db-sync follows the +chain through a node socket and writes a queryable Postgres schema. + +This guide covers the two decisions you make before applying: where the Postgres +database comes from, and where db-sync runs relative to the network. For the full +spec and every `insert` option, see the +[CardanoDBSync reference](../reference/cardanodbsync.md). For the rationale behind +the placement and insert trade-offs, see +[Architecture](../concepts/architecture.md). + +## Before you start + +- A `CardanoNetwork` already exists and is `Ready` in the namespace you will use. + See [Networks](../developer/networks.md) and the + [yacd CLI reference](../reference/cli.md). +- You can apply manifests to that namespace with `kubectl`. + +`CardanoDBSync` requires only `networkRef.name` and a `database` mode; every other +field has a default. The examples below use the `devnet` network in the +`yacd-smoke` namespace. + +## Decision 1: choose the Postgres source + +`spec.database` requires **exactly one** of `managed` or `external`. + +### Managed Postgres (recommended for local development) + +Set `database.managed` and the controller provisions Postgres for you. When you do +not supply credentials, it creates an owned Secret holding the generated password +and reports the name in `status.database.authSecretName`. + +```yaml +spec: + networkRef: + name: devnet + database: + managed: {} +``` + +`managed: {}` accepts all defaults (image `postgres:17.2-alpine`, database +`cexplorer`, user `postgres`). To reuse a password you already hold, point +`managed.authSecretRef.name` at a same-namespace Secret with the password under +the key `password`. See the +[CardanoDBSync reference](../reference/cardanodbsync.md) for the full `managed` +surface. + +### External Postgres + +Set `database.external` to point db-sync at a Postgres instance you operate. You +must supply the password through `passwordSecretRef`, which references a +same-namespace Secret. + +Create the Secret first: + +```sh +kubectl -n yacd-smoke create secret generic dbsync-postgres \ + --from-literal=password=change-me +``` + +Then reference it from the `CardanoDBSync`: + +```yaml +spec: + networkRef: + name: devnet + database: + external: + host: postgres.yacd-smoke.svc.cluster.local + passwordSecretRef: + name: dbsync-postgres +``` + +`passwordSecretRef.key` defaults to `password`, so a Secret with that key needs no +`key` field. `port` (`5432`), `database` (`cexplorer`), `user` (`postgres`), and +`sslMode` (`disable`) all default; override them in `external` when your server +differs. See the +[CardanoDBSync reference](../reference/cardanodbsync.md) for the full `external` +surface, including `sslMode` values. + +## Decision 2: choose where db-sync runs + +`spec.placement.mode` selects how db-sync reaches the node socket. + +- `dedicatedFollower` (default) runs a separate follower node colocated with + db-sync, owned by the `CardanoDBSync` controller. Omitting `placement` keeps this + behavior. +- `primarySidecar` injects db-sync into the referenced network's primary node Pod + instead of running a separate follower. + +```yaml +spec: + placement: + mode: primarySidecar +``` + +`followerNode` cannot be set when `placement.mode` is `primarySidecar`; the spec +rejects that combination. For when to prefer each mode, see +[Architecture](../concepts/architecture.md). + +!!! warning "Placement is fixed for a database" + The placement mode is recorded against the db-sync state in + `status.database.acceptedPlacementMode`. Changing it later requires deleting + and recreating the `CardanoDBSync` with a fresh or compatible database. + +## Apply the CardanoDBSync + +Use one of the manifests from [Recipes](../recipes.md) (mirrored from `examples/`), +or write your own from the decisions above. Apply it into the network's namespace: + +```sh +kubectl apply -f cardanodbsync.yaml +``` + +## Check Ready and Synced + +`CardanoDBSync` has no dedicated `yacd` subcommand; check it with `kubectl`. The +printed columns surface progress at a glance: + +```sh +kubectl -n yacd-smoke get cardanodbsync +``` + +```text +NAME NETWORK READY SYNCED AGE +dbsync devnet True True 5m +``` + +- `READY` (`Ready` condition) reports that db-sync is usable through its published + database endpoint. +- `SYNCED` (`Synced` condition) reports that db-sync has caught up to the node tip. + +Wait for both conditions to read `True`: + +```sh +kubectl -n yacd-smoke wait --for=condition=Ready cardanodbsync/dbsync --timeout=15m +kubectl -n yacd-smoke wait --for=condition=Synced cardanodbsync/dbsync --timeout=60m +``` + +For more detail, inspect the resource status. It reports the Postgres endpoint, +indexing progress (block heights and lag), and, for managed Postgres without your +own Secret, the generated credentials Secret name: + +```sh +kubectl -n yacd-smoke get cardanodbsync/dbsync -o yaml +``` + +The Postgres endpoint and metrics endpoint appear under `status.endpoints`. For +the complete status surface, including every condition type, see the +[CardanoDBSync reference](../reference/cardanodbsync.md). + +## Next steps + +- Tune what db-sync writes with `spec.config.insert`; see the + [CardanoDBSync reference](../reference/cardanodbsync.md). +- Understand the placement and insert trade-offs in + [Architecture](../concepts/architecture.md). +- Read the upstream + [cardano-db-sync documentation](https://github.com/IntersectMBO/cardano-db-sync) + for schema details and query guidance. diff --git a/docs/operator/installation.md b/docs/operator/installation.md new file mode 100644 index 0000000..6f4e85d --- /dev/null +++ b/docs/operator/installation.md @@ -0,0 +1,152 @@ +# Installation + +The YACD operator can be installed two equivalent ways: with the `yacd` CLI, which renders and applies the bundled chart directly (no Helm required), or with [Helm](https://helm.sh) from the OCI chart published to GitHub Container Registry. Either way deploys the controller manager, its RBAC and ServiceAccount, a secured metrics Service, and the `CardanoNetwork` and `CardanoDBSync` CRDs. + +This page covers installing onto an existing cluster. For a local [k3d](https://k3d.io) cluster, the `yacd` CLI manages its own cluster lifecycle — `yacd devnet` installs the operator for you; see the [CLI reference](../reference/cli.md). For every chart value and manager flag, see the [configuration reference](../reference/configuration.md). + +## Prerequisites + +- A Kubernetes cluster, version 1.29 or later (the chart sets `kubeVersion: ">= 1.29.0-0"`). +- `kubectl` configured against the target cluster. +- For `yacd install`: the [`yacd` CLI](../reference/cli.md#install) on your `PATH` (no Helm needed). +- For the Helm path: Helm 3.8 or later, which can pull OCI charts. + +## Install + +=== "yacd install" + + `yacd install` renders the operator's bundled chart in memory and applies it to the cluster your kubeconfig points at. It installs the operator when absent and upgrades it otherwise, then waits for the manager to become ready. + + ```sh + yacd install --namespace + ``` + + The namespace defaults to `yacd-system` and is created if it does not exist. `--wait` (default true) blocks until the manager Deployment is Available, up to `--timeout` (default `5m`). Preview the action without changing the cluster with `--dry-run`. The operator version is the one this CLI embeds; to move it, upgrade the CLI. Pass operational value overrides with `-f`/`--set`/`--set-string` — see the [CLI reference](../reference/cli.md#install). + +=== "Helm" + + Install the chart into a namespace, creating it if it does not exist. Replace `` with the release version you want (for example `0.2.1`) and `` with your target namespace. + + ```sh + helm install yacd oci://ghcr.io/meigma/yacd/chart \ + --version \ + --namespace \ + --create-namespace + ``` + + The chart name is `chart` and its version tracks the release version without the leading `v` (the published chart `0.2.1` ships `appVersion` `v0.2.1`). To inspect the chart metadata before installing: + + ```sh + helm show chart oci://ghcr.io/meigma/yacd/chart --version + ``` + + Override values with `--set` or `-f values.yaml`. + +Both methods bundle the CRDs `cardanonetworks.yacd.meigma.io` and `cardanodbsyncs.yacd.meigma.io`, so you do not apply CRDs separately. The full value surface (image, metrics, RBAC, leader election, manager logging, Kyverno, security context, scheduling) lives in the [configuration reference](../reference/configuration.md). + +## Verify + +Confirm the manager Deployment becomes Available: + +```sh +kubectl rollout status deployment/yacd-controller-manager --namespace +``` + +Confirm the CRDs are registered: + +```sh +kubectl get crd cardanonetworks.yacd.meigma.io cardanodbsyncs.yacd.meigma.io +``` + +Operators drive networks with the same `yacd` verbs documented in the [CLI reference](../reference/cli.md). Once the manager is Available you can apply a `CardanoNetwork` or `CardanoDBSync`; see the copy-paste manifests in [Recipes](../recipes.md). + +## Upgrading + +=== "yacd install" + + Re-run `yacd install` against the cluster. It upgrades an older same-major install and re-applies an equal version to heal drift; it refuses a newer or major-mismatched in-cluster version with actionable guidance. Because the operator version is pinned to the CLI, **upgrade the CLI to move the operator version**, then re-run: + + ```sh + yacd install --namespace + ``` + +=== "Helm" + + Upgrade to a new chart version in place: + + ```sh + helm upgrade yacd oci://ghcr.io/meigma/yacd/chart \ + --version \ + --namespace + ``` + +!!! warning "Helm does not upgrade bundled CRDs (Helm path only)" + `yacd install` re-renders the CRDs from the embedded chart, so they travel with the CLI version. Helm, by contrast, installs the CRDs under the chart's `crds/` directory on first install but **never upgrades or deletes them** on `helm upgrade` or `helm uninstall`. When a release changes a CRD schema, apply the updated definitions yourself before or after the Helm upgrade: + + ```sh + kubectl apply -f https://raw.githubusercontent.com/meigma/yacd/v/charts/yacd/crds/yacd.meigma.io_cardanonetworks.yaml + kubectl apply -f https://raw.githubusercontent.com/meigma/yacd/v/charts/yacd/crds/yacd.meigma.io_cardanodbsyncs.yaml + ``` + + Review the release notes for the target version to confirm whether a CRD changed before applying. + +## Metrics + +The chart ships a secured metrics Service by default. The manager serves metrics over HTTPS (`--metrics-secure` defaults to true) behind the Kubernetes authn/authz filter, so scrapers must authenticate and carry RBAC to read the endpoint. The chart also creates a `ClusterRole` that grants the metrics-reader permission for that path. + +The Service is a `ClusterIP` named `yacd-controller-manager-metrics-service`, exposing port `8443` by default. To scrape it, grant your scraper the metrics-reader role and target the Service over HTTPS. To turn metrics off or change the port and TLS material, see the `metrics.*` values in the [configuration reference](../reference/configuration.md). + +## Supply-chain image verification (optional) + +The release workflow attests the manager image and Helm chart with GitHub-native (Sigstore keyless) attestations. The chart includes an opt-in [Kyverno](https://kyverno.io) `ClusterPolicy` that verifies those attestations on admission. It is disabled by default (`kyverno.imageVerification.enabled: false`). + +Enabling it requires a running Kyverno installation in the cluster. When enabled with no explicit `imageReferences`, the policy verifies the configured manager image repository, requiring a keyless Sigstore attestation from the `meigma/yacd` release workflow. To enable it: + +=== "yacd install" + + ```sh + yacd install --namespace \ + --set kyverno.imageVerification.enabled=true + ``` + +=== "Helm" + + ```sh + helm upgrade yacd oci://ghcr.io/meigma/yacd/chart \ + --version \ + --namespace \ + --set kyverno.imageVerification.enabled=true + ``` + +The attestor issuer, subject pattern, Rekor URL, validation failure action, and image references are all configurable; see the `kyverno.imageVerification.*` values in the [configuration reference](../reference/configuration.md). To verify a pulled chart or image directly with the GitHub CLI instead of at admission time, use `gh attestation verify`. + +## Uninstall + +!!! note "`yacd uninstall` is not yet available" + Removing the operator is a manual step today; the path depends on how you installed it. + +=== "yacd install" + + Delete the install namespace (which removes the manager Deployment, ServiceAccount, Services, and namespaced RBAC) and the chart's cluster-scoped RBAC: + + ```sh + kubectl delete namespace + kubectl delete clusterrole,clusterrolebinding -l app.kubernetes.io/name=yacd + ``` + +=== "Helm" + + Remove the release: + + ```sh + helm uninstall yacd --namespace + ``` + +!!! warning "CRDs and custom resources are not removed" + Neither path deletes the CRDs or any `CardanoNetwork` or `CardanoDBSync` resources you created. Delete those resources first if you want the operator to tear down their owned runtime children, then remove the CRDs once nothing depends on them: + + ```sh + kubectl delete cardanonetworks --all --all-namespaces + kubectl delete cardanodbsyncs --all --all-namespaces + kubectl delete crd cardanonetworks.yacd.meigma.io cardanodbsyncs.yacd.meigma.io + ``` diff --git a/docs/operator/testing.md b/docs/operator/testing.md new file mode 100644 index 0000000..4c00b1f --- /dev/null +++ b/docs/operator/testing.md @@ -0,0 +1,110 @@ +# Testing & CI + +Use YACD to give an automated test suite a real, ephemeral Cardano network and +tear it down deterministically afterward. The pattern is three commands: + +```sh +yacd up my-net -f env.yaml --wait # create, block until Ready +yacd run my-net -- # run tests with YACD_* wired in +yacd down my-net --wait # delete, block until gone +``` + +Your test runner stays YACD-agnostic: it reads the `YACD_*` environment +variables `run` injects and never parses a YACD file, so the same suite works on +a laptop and in CI. For the meaning of those variables and the `run`/`exec` +semantics, see [Host access and the `YACD_*` contract](../developer/connecting-tools.md); +for every flag and default, see the [CLI reference](../reference/cli.md). + +## 1. Bring the network up, gated on Ready + +`yacd up` server-side-applies the `CardanoNetwork` rendered from your +environment file and, with `--wait` (the default), blocks until the network +reports Ready before returning a zero exit code. A non-zero exit means the +network never became Ready, so a CI step can depend on `up` succeeding: + +```sh +yacd up my-net -f env.yaml --wait +``` + +`--wait` is on by default; pass `--wait=false` to apply without blocking. The +readiness deadline is `--timeout` (default 12m). Tune it for slow runners or +larger profiles; raising it does not change what "Ready" means, only how long +`up` will wait for it. See the [CLI reference](../reference/cli.md) for the full +flag set. + +## 2. Run the suite under `yacd run` + +`yacd run NAME -- ` establishes scoped port-forwards to the network's +chain APIs, exports the `YACD_*` environment into the command, runs it on the +host, and tears the forwards down when it exits. The command's exit status is +propagated to your shell, so the test runner's pass/fail code is what CI sees: + +```sh +yacd run my-net -- go test ./e2e/... +``` + +Always put `--` before the test command so its own flags are passed through to +it rather than parsed by `yacd`. Inside the command, the runner reads endpoints +from `YACD_OGMIOS_URL` and `YACD_KUPO_URL` (loopback URLs on the host). The +variable names and loopback details are documented once in +[Host access and the `YACD_*` contract](../developer/connecting-tools.md). + +!!! warning "Use `exec`, not `run`, for `cardano-cli`" + `run` forwards Ogmios, Kupo, and the faucet over TCP. Tools that reach the + node over its local Unix socket (notably `cardano-cli`) need + `yacd exec NAME -- …` instead, which runs in the node Pod and sets + `CARDANO_NODE_SOCKET_PATH`. See + [Choose a verb](../developer/connecting-tools.md#choose-a-verb). + +## 3. Tear down deterministically + +`yacd down` deletes the `CardanoNetwork` and, with `--wait` (the default), +blocks until the object and its garbage-collected children are gone before +returning. Deletion is idempotent: a network that is already absent is reported +as success, so teardown is safe to run unconditionally in a cleanup step. + +```sh +yacd down my-net --wait +``` + +The removal deadline is `--timeout` (default 5m). Run teardown even when the +test step failed, so a broken run never leaks a cluster network. + +## Minimal CI snippet + +The three steps map directly onto a CI job. Bring the network up, run the suite, +and tear down whether or not the suite passed. + +```yaml +steps: + - name: Bring up the network + run: yacd up my-net -f env.yaml --wait + + - name: Run the test suite + run: yacd run my-net -- go test ./e2e/... + + - name: Tear down the network + if: always() + run: yacd down my-net --wait +``` + +`if: always()` (or your CI's equivalent) guarantees teardown runs after a +failed test step. Because `down` is idempotent, the cleanup step is also safe to +keep even if `up` never created the network. + +## Selecting the cluster and namespace + +`up`, `run`, and `down` all accept the global `--kubeconfig`, `--context`, and +`-n/--namespace` flags, so a CI runner can target a cluster without editing the +test command. These globals are documented once in the +[CLI reference](../reference/cli.md). + +When `--namespace` is not set, the namespace defaults to the network name, so +`yacd up my-net …` applies the network into the `my-net` namespace and `up` +auto-creates that namespace if it does not exist. + +!!! warning "Mainnet is not a CI target" + `yacd up` refuses to apply a mainnet network without `--allow-mainnet`, + because mainnet creates large persistent volumes and bootstraps from + Mithril. Automated tests should use a local devnet or a public testnet + profile, not mainnet. diff --git a/docs/recipes.md b/docs/recipes.md new file mode 100644 index 0000000..b535fb8 --- /dev/null +++ b/docs/recipes.md @@ -0,0 +1,243 @@ +# Recipes + +Copy-paste manifests for the most common YACD setups. Each recipe is mirrored +verbatim from a file under `examples/`. Use them as a starting point and adjust +field values to suit your cluster. + +!!! tip + `yacd init` prints a commented version of the single-pool local devnet below, + ready to redirect into a file (`yacd init > yacd.yaml`). It is the quickest + way to start a custom network; see [Defining networks](developer/networks.md). + +Developer environment files (`kind: Environment`) are applied with the CLI: + +```sh +yacd up dev -f yacd.yaml +``` + +DB-Sync resources (`kind: CardanoDBSync`) are applied with `kubectl`: + +```sh +kubectl apply -f cardanodbsync.yaml +``` + +For the full field semantics, defaults, and enums behind these manifests, see +the reference pages linked under each section. For every CLI flag and default, +see [CLI reference](reference/cli.md). + +## Local networks + +A local network runs a self-contained devnet with a fast block schedule. Every +local network is created with a genesis-funded `faucet` wallet that you fund +developer wallets from with [`yacd wallet`](developer/funding.md). See the +[Environment file reference](reference/environment.md) for every field. + +### Single-pool local devnet + +Use when you want an isolated, fast local Cardano network. + +```yaml +apiVersion: yacd.meigma.io/devconfig/v1alpha1 +kind: Environment +spec: + network: + mode: local + node: + version: "11.0.1" + port: 3001 + storage: + size: 2Gi + # Ogmios and Kupo are enabled by default. A local network is automatically + # given a genesis-funded `faucet` wallet — the network's funded wallet and + # the default source for `yacd wallet topup`. + local: + networkMagic: 42 + era: conway + timing: + slotLength: 100ms + epochLength: 500 + topology: + pools: + count: 1 +``` + +## Public networks + +A public network syncs a node against an upstream Cardano network. Select the +target with `spec.network.public.profile`. See the +[Environment file reference](reference/environment.md) for every field. + +### Preview testnet + +Use when you want to develop against the public Preview testnet. + +```yaml +apiVersion: yacd.meigma.io/devconfig/v1alpha1 +kind: Environment +spec: + network: + mode: public + node: + version: "11.0.1" + port: 3001 + storage: + size: 20Gi + public: + profile: preview +``` + +### Preprod testnet + +Use when you want to develop against the public Preprod testnet. + +```yaml +apiVersion: yacd.meigma.io/devconfig/v1alpha1 +kind: Environment +spec: + network: + mode: public + node: + version: "11.0.1" + port: 3001 + storage: + size: 20Gi + public: + profile: preprod +``` + +### Mainnet + +Use when you need to sync against Cardano mainnet. + +```yaml +apiVersion: yacd.meigma.io/devconfig/v1alpha1 +kind: Environment +spec: + network: + mode: public + node: + version: "11.0.1" + port: 3001 + public: + profile: mainnet + bootstrap: + mithril: {} +``` + +!!! warning "Mainnet has hard requirements" + - A [Mithril](https://mithril.network) bootstrap (`bootstrap.mithril`) is + required to seed the chain database; syncing mainnet from genesis is not + practical. + - Mainnet needs **large persistent storage** well beyond the 20Gi used for + testnets. Size `node.storage.size` for the full mainnet chain. + - The CLI refuses to apply a mainnet network unless you pass + `--allow-mainnet`. See [CLI reference](reference/cli.md). + +## DB-Sync + +[cardano-db-sync](https://github.com/IntersectMBO/cardano-db-sync) indexes a +network's chain into PostgreSQL. A `CardanoDBSync` references an existing +network through `spec.networkRef` and connects to either an operator-managed or +an external database. See the +[CardanoDBSync reference](reference/cardanodbsync.md) for every field. + +### Managed PostgreSQL + +Use when you want YACD to provision and own the PostgreSQL database for you. + +```yaml +apiVersion: yacd.meigma.io/v1alpha1 +kind: CardanoDBSync +metadata: + name: dbsync + namespace: yacd-smoke +spec: + networkRef: + name: devnet + database: + managed: {} +``` + +### External PostgreSQL + +Use when you connect DB-Sync to a PostgreSQL instance you manage yourself, +supplying the password through a `Secret`. + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: dbsync-postgres + namespace: yacd-smoke +type: Opaque +stringData: + password: change-me +--- +apiVersion: yacd.meigma.io/v1alpha1 +kind: CardanoDBSync +metadata: + name: dbsync + namespace: yacd-smoke +spec: + networkRef: + name: devnet + database: + external: + host: postgres.yacd-smoke.svc.cluster.local + passwordSecretRef: + name: dbsync-postgres +``` + +### Preprod DB-Sync on a dedicated follower + +Use when you index a public Preprod network with a dedicated follower node and +a minimal insert preset to reduce write volume. + +```yaml +apiVersion: yacd.meigma.io/v1alpha1 +kind: CardanoDBSync +metadata: + name: preprod-dbsync + namespace: yacd-smoke +spec: + networkRef: + name: preprod-smoke + placement: + mode: dedicatedFollower + database: + managed: {} + config: + insert: + preset: disable_all +``` + +### Preview DB-Sync as a primary sidecar + +Use when you co-locate DB-Sync with the primary node as a sidecar and tune its +runtime for a lightweight, low-overhead index. + +```yaml +apiVersion: yacd.meigma.io/v1alpha1 +kind: CardanoDBSync +metadata: + name: preview-dbsync-sidecar + namespace: yacd-smoke +spec: + networkRef: + name: preview-smoke + placement: + mode: primarySidecar + database: + managed: + parameters: + maxParallelMaintenanceWorkers: 0 + config: + ledgerBackend: inmemory + runtime: + cache: false + epochTable: false + forceIndexes: false + metricsPort: 8080 + insert: + preset: disable_all +``` diff --git a/docs/reference/cardanodbsync.md b/docs/reference/cardanodbsync.md new file mode 100644 index 0000000..3261865 --- /dev/null +++ b/docs/reference/cardanodbsync.md @@ -0,0 +1,329 @@ +# CardanoDBSync + +`CardanoDBSync` indexes a [`CardanoNetwork`](cardanonetwork.md) into a Postgres +database using [cardano-db-sync](https://github.com/IntersectMBO/cardano-db-sync). +This page is the dry, complete field reference for the custom resource. + +| Property | Value | +| --- | --- | +| API group / version | `yacd.meigma.io/v1alpha1` | +| Kind | `CardanoDBSync` | +| List kind | `CardanoDBSyncList` | +| Plural | `cardanodbsyncs` | +| Singular | `cardanodbsync` | +| Scope | Namespaced | +| Status subresource | enabled | + +For copy-paste manifests, see [Recipes](../recipes.md). The objects that +`CardanoDBSync` references (Secrets, the `CardanoNetwork`) must live in the same +namespace. + +## Print columns + +`kubectl get cardanodbsyncs` shows: + +| Column | Source | +| --- | --- | +| `Network` | `.spec.networkRef.name` | +| `Ready` | `.status.conditions[?(@.type=="Ready")].status` | +| `Synced` | `.status.conditions[?(@.type=="Synced")].status` | +| `Age` | `.metadata.creationTimestamp` | + +## Spec + +`spec` is required. + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `networkRef` | object | yes | — | Same-namespace `CardanoNetwork` that db-sync indexes. See [networkRef](#networkref). | +| `image` | string | yes | `ghcr.io/intersectmbo/cardano-db-sync:13.7.1.0` | cardano-db-sync image reference. | +| `resources` | object | no | — | db-sync container resources (core `ResourceRequirements`). | +| `placement` | object | no | — | Where db-sync runs relative to the network. When omitted, the controller preserves the dedicated follower workload. See [placement](#placement). | +| `followerNode` | object | no | — | Dedicated follower node colocated with db-sync for local node socket access. See [followerNode](#followernode). | +| `database` | object | yes | — | Postgres database used by db-sync. Exactly one mode must be set. See [database](#database). | +| `stateStorage` | object | no | — | Persistent storage for db-sync ledger state. See [storage](#storage-spec). | +| `config` | object | no | — | Upstream db-sync behavior in Kubernetes-style field names. See [config](#config). | + +### Spec-level validation + +| Rule | Message | +| --- | --- | +| `!has(self.placement) \|\| self.placement.mode != 'primarySidecar' \|\| !has(self.followerNode)` | `followerNode cannot be set when placement.mode is primarySidecar` | + +### networkRef + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `name` | string | yes | — | Name of the referenced `CardanoNetwork`. Minimum length 1. | + +### placement + +| Field | Type | Required | Default | Enum | Description | +| --- | --- | --- | --- | --- | --- | +| `mode` | string | yes | `dedicatedFollower` | `dedicatedFollower`, `primarySidecar` | Whether db-sync uses a dedicated follower node or asks the referenced network's primary Pod to host it as a sidecar. | + +`dedicatedFollower` keeps the two-container workload with a colocated follower +node owned by the `CardanoDBSync` controller. `primarySidecar` requests db-sync +placement inside the referenced `CardanoNetwork` primary node Pod; `followerNode` +must not be set in this mode. + +### followerNode + +Configures the follower node owned by db-sync. Only meaningful in +`dedicatedFollower` placement. + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `image` | string | no | derived from the referenced `CardanoNetwork` | Overrides the follower cardano-node image. | +| `storage` | object | no | — | Persistent follower node database storage. See [storage](#storage-spec). | +| `resources` | object | no | — | Follower node container resources (core `ResourceRequirements`). | + +### database + +Exactly one of `external` or `managed` must be set. + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `external` | object | no | — | References a Postgres instance managed outside this resource. See [database.external](#databaseexternal). | +| `managed` | object | no | — | YACD-managed Postgres for local development. See [database.managed](#databasemanaged). | + +| Rule | Message | +| --- | --- | +| `has(self.external) != has(self.managed)` | `exactly one of database.external or database.managed must be set` | + +#### database.external + +| Field | Type | Required | Default | Enum | Description | +| --- | --- | --- | --- | --- | --- | +| `host` | string | yes | — | — | DNS name or IP address of the Postgres server. Minimum length 1. | +| `port` | int32 | yes | `5432` | — | Postgres server port. Range 1–65535. | +| `database` | string | yes | `cexplorer` | — | Postgres database name. | +| `user` | string | yes | `postgres` | — | Postgres user name. | +| `passwordSecretRef` | object | yes | — | — | Same-namespace Secret holding the Postgres password. See [secret key reference](#secret-key-reference). | +| `sslMode` | string | yes | `disable` | `disable`, `require`, `verify-ca`, `verify-full` | libpq `sslmode` used for Postgres connections. | + +#### database.managed + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `image` | string | yes | `postgres:17.2-alpine` | Postgres image reference. | +| `database` | string | yes | `cexplorer` | Postgres database name. | +| `user` | string | yes | `postgres` | Postgres user name. | +| `authSecretRef` | object | no | controller creates an owned Secret | Same-namespace Secret with the Postgres password in key `password`. When omitted, the controller creates a Secret and reports its name in `status.database.authSecretName`. See [secret reference](#secret-reference). | +| `storage` | object | no | — | Persistent Postgres data storage. See [storage](#storage-spec). | +| `parameters` | object | no | — | Basic Postgres startup parameters. See [database.managed.parameters](#databasemanagedparameters). | +| `resources` | object | no | — | Postgres container resources (core `ResourceRequirements`). | + +##### database.managed.parameters + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `maintenanceWorkMem` | quantity | no | — | Sets Postgres `maintenance_work_mem`. | +| `maxParallelMaintenanceWorkers` | int32 | no | — | Sets Postgres `max_parallel_maintenance_workers`. Minimum 0. | + +### storage (spec) + +The same `storage` shape is used by `spec.stateStorage`, +`spec.followerNode.storage`, and `spec.database.managed.storage`. + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `size` | quantity | no | — | Requested persistent volume size. | +| `storageClassName` | string | no | — | Kubernetes StorageClass used for the PVC. | + +### Secret references + +#### secret reference + +Used by `database.managed.authSecretRef`. + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `name` | string | yes | — | Name of the referenced Secret. Minimum length 1. | + +#### secret key reference + +Used by `database.external.passwordSecretRef`. + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `name` | string | yes | — | Name of the referenced Secret. Minimum length 1. | +| `key` | string | yes | `password` | Secret data key containing the value. Minimum length 1. | + +## config + +Configures upstream db-sync behavior in Kubernetes-style field names. The +controller translates this object into the upstream db-sync configuration file. + +| Field | Type | Required | Default | Enum | Description | +| --- | --- | --- | --- | --- | --- | +| `runtime` | object | no | — | — | db-sync runtime flags outside `insert_options`. See [config.runtime](#configruntime). | +| `ledgerBackend` | string | yes | `lsm` | `inmemory`, `lsm` | How db-sync stores its ledger UTxO set. `inmemory` keeps it in memory; `lsm` stores it on disk using LSM trees. | +| `snapshot` | object | no | — | — | Ledger state snapshot behavior. See [config.snapshot](#configsnapshot). | +| `insert` | object | no | — | — | Upstream `insert_options`. See [config.insert](#configinsert). | +| `ipfsGateways` | array of string | no | — | — | Gateways used for offchain metadata fetching. | + +When `config` is provided, `ledgerBackend` is required within it. + +### config.runtime + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `cache` | bool | yes | `true` | Whether db-sync caches are enabled. | +| `epochTable` | bool | yes | `true` | Whether db-sync populates the epoch table. | +| `forceIndexes` | bool | yes | `false` | Whether db-sync creates indexes at startup rather than later in the sync lifecycle. | +| `metricsPort` | int32 | yes | `8080` | db-sync Prometheus metrics port. Range 1–65535. | + +### config.snapshot + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `nearTipEpoch` | int64 | yes | `580` | Epoch threshold where db-sync considers itself near tip for snapshot frequency. Minimum 0. | + +### config.insert + +Maps to upstream db-sync `insert_options`. `preset` selects a profile; the +explicit fields below are interpreted by the controller as overrides on top of +the preset. + +| Field | Type | Required | Default | Enum | Description | +| --- | --- | --- | --- | --- | --- | +| `preset` | string | yes | `full` | `full`, `only_utxo`, `only_governance`, `disable_all` | Upstream insert profile. `full` enables the normal full schema surface; `only_utxo` keeps only UTxO-oriented data; `only_governance` keeps governance-oriented data; `disable_all` keeps only the minimum block and tx data. | +| `txCbor` | bool | no | — | — | Transaction CBOR collection. | +| `txOut` | object | no | — | — | Transaction output storage. See [config.insert.txOut](#configinserttxout). | +| `ledger` | string | no | — | `enable`, `disable`, `ignore` | Ledger state maintenance and use. `enable` maintains and uses ledger-derived data; `disable` avoids maintaining ledger state; `ignore` maintains state but does not use its data. | +| `shelley` | object | no | — | — | Shelley-era table inserts. See [config.insert.shelley](#configinsertshelley). | +| `multiAsset` | object | no | — | — | Multi-asset table inserts. See [config.insert.multiAsset](#configinsertmultiasset). | +| `metadata` | object | no | — | — | Transaction metadata inserts. See [config.insert.metadata](#configinsertmetadata). | +| `plutus` | object | no | — | — | Plutus and script table inserts. See [config.insert.plutus](#configinsertplutus). | +| `governance` | bool | no | — | — | Governance-related data inserts. | +| `offchainPoolData` | bool | no | — | — | Stake pool offchain metadata fetching. | +| `offchainVoteData` | bool | no | — | — | Governance offchain metadata fetching. | +| `poolStats` | bool | no | — | — | Pool stats inserts. | +| `jsonType` | string | no | — | `text`, `jsonb`, `disable` | Upstream `json_type` option. `text` stores JSON as text; `jsonb` stores it as jsonb; `disable` disables JSON storage where supported. | +| `removeJsonbFromSchema` | bool | no | — | — | Whether db-sync removes jsonb data types from affected schema columns. | + +#### config.insert.txOut + +| Field | Type | Required | Default | Enum | Description | +| --- | --- | --- | --- | --- | --- | +| `mode` | string | no | — | `enable`, `disable`, `consumed`, `prune`, `bootstrap` | Upstream `tx_out` value. `enable` stores all inputs and outputs; `disable` disables the tx input/output tables; `consumed` stores `consumed_by_tx_id` for direct UTxO queries; `prune` periodically prunes consumed outputs; `bootstrap` delays UTxO insertion until db-sync reaches the tip. | +| `forceTxIn` | bool | no | — | — | Keeps `tx_in` populated for `consumed`, `prune`, or `bootstrap` modes. | +| `useAddressTable` | bool | no | — | — | Enables the normalized address table schema variant. | + +#### config.insert.shelley + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `enabled` | bool | no | — | Shelley-era data inserts. | +| `stakeAddresses` | array of string | no | — | Limits Shelley data to specific stake addresses. | + +#### config.insert.multiAsset + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `enabled` | bool | no | — | Multi-asset data inserts. | +| `policies` | array of string | no | — | Limits multi-asset data to specific policy hashes. | + +#### config.insert.metadata + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `enabled` | bool | no | — | Transaction metadata inserts. | +| `keys` | array of int64 | no | — | Limits metadata inserts to specific numeric labels. | + +#### config.insert.plutus + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `enabled` | bool | no | — | Plutus and script data inserts. | +| `scriptHashes` | array of string | no | — | Limits Plutus data to specific script hashes. | + +## Status + +All status fields are optional and populated by the controller. + +| Field | Type | Description | +| --- | --- | --- | +| `observedGeneration` | int64 | Most recent generation observed by the controller. | +| `endpoints` | object | Cluster-local connection details. See [status.endpoints](#statusendpoints). | +| `database` | object | Database-specific runtime details. See [status.database](#statusdatabase). | +| `sync` | object | db-sync indexing progress. See [status.sync](#statussync). | +| `placement` | object | Effective placement mode and primary-sidecar contract. See [status.placement](#statusplacement). | +| `conditions` | array | Resource conditions. See [Conditions](#conditions). | + +### status.endpoints + +| Field | Type | Description | +| --- | --- | --- | +| `postgres` | object | Postgres endpoint used by db-sync and clients. | +| `metrics` | object | db-sync Prometheus metrics endpoint. | + +Each endpoint object (`ServiceEndpointStatus`) contains: + +| Field | Type | Description | +| --- | --- | --- | +| `serviceName` | string | Kubernetes Service name. | +| `port` | int32 | Service port. | +| `url` | string | Convenience URL for protocols with a stable URL shape. | + +### status.database + +| Field | Type | Enum | Description | +| --- | --- | --- | --- | +| `acceptedIdentityFingerprint` | string | — | Database-affecting plan identity the controller accepted on owned runtime material. Mirrors the value from the db-sync state PVC annotation; not the authority for identity validation. | +| `acceptedPlacementMode` | string | `dedicatedFollower`, `primarySidecar` | Placement mode accepted for the current db-sync state. Changing it requires deleting and recreating the resource with a fresh or compatible database. | +| `authSecretName` | string | — | Same-namespace Secret with generated database credentials when the user did not provide `authSecretRef`. | + +### status.sync + +| Field | Type | Description | +| --- | --- | --- | +| `nodeBlockHeight` | int64 | Latest block height reported by the follower node. | +| `dbBlockHeight` | int64 | Latest block height inserted into Postgres. | +| `dbSlotHeight` | int64 | Latest slot inserted into Postgres. | +| `dbQueueLength` | int64 | Current db-sync database event queue length. | +| `lagBlocks` | int64 | Difference between `nodeBlockHeight` and `dbBlockHeight`. | +| `epoch` | int64 | Latest epoch observed by db-sync. | + +### status.placement + +| Field | Type | Enum | Description | +| --- | --- | --- | --- | +| `mode` | string | `dedicatedFollower`, `primarySidecar` | Effective placement mode for this reconcile. | +| `primarySidecar` | object | — | Attachable material contract published when `SidecarMaterialReady=True`. | + +`primarySidecar` contains: + +| Field | Type | Description | +| --- | --- | --- | +| `networkName` | string | Referenced `CardanoNetwork` name this sidecar material is valid for. | +| `revision` | string | Opaque sha256 rollout revision over all sidecar-mounted material. | +| `resources` | object | `CardanoDBSync`-owned resources mounted by the primary Pod sidecar. | + +`primarySidecar.resources` contains: + +| Field | Type | Description | +| --- | --- | --- | +| `configMapName` | string | db-sync configuration ConfigMap name. | +| `pgpassSecretName` | string | db-sync pgpass Secret name. | +| `statePVCName` | string | db-sync state PVC name. | +| `metricsServiceName` | string | db-sync metrics Service name. | + +### Conditions + +`status.conditions` is a list of standard `metav1.Condition` objects keyed by +`type`. Each entry has `type`, `status` (`True`, `False`, or `Unknown`), +`reason`, `message`, `lastTransitionTime`, and `observedGeneration`. + +| Type | Meaning | +| --- | --- | +| `Ready` | db-sync is usable through its published database endpoint. | +| `FollowerNodeReady` | The colocated follower node is running. | +| `NodeSocketReady` | The node socket used by db-sync is reachable. | +| `SidecarMaterialReady` | Primary-sidecar mounted material is attachable. | +| `PostgresReady` | Postgres is running and accepting local connections. | +| `DBSyncReady` | The db-sync process is running. | +| `Synced` | db-sync has caught up to the node tip. | +| `Progressing` | The resource is being created or updated. | +| `Degraded` | The resource failed to reach or maintain desired state. | diff --git a/docs/reference/cardanonetwork.md b/docs/reference/cardanonetwork.md new file mode 100644 index 0000000..f6c963a --- /dev/null +++ b/docs/reference/cardanonetwork.md @@ -0,0 +1,319 @@ +# CardanoNetwork + +`CardanoNetwork` is the YACD custom resource that declares a single Cardano +development network: one primary [cardano-node](https://developers.cardano.org) +workload plus its chain APIs ([Ogmios](https://ogmios.dev) and +[Kupo](https://cardanosolutions.github.io/kupo/)). + +| | | +| --- | --- | +| API group/version | `yacd.meigma.io/v1alpha1` | +| Kind | `CardanoNetwork` | +| List kind | `CardanoNetworkList` | +| Plural | `cardanonetworks` | +| Singular | `cardanonetwork` | +| Scope | Namespaced | +| Status subresource | Enabled | + +A network is created in one of two **modes**: `local` (YACD generates and owns a +fresh devnet) or `public` (the node joins a known public profile). Exactly one of +`spec.local` or `spec.public` must be present, matching `spec.mode`. + +Copy-paste manifests live in [recipes](../recipes.md). The companion CLI and its +flags are documented in the [CLI reference](cli.md). For the sibling resource +that attaches cardano-db-sync, see +[CardanoDBSync](cardanodbsync.md). + +## TypeMeta and metadata + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `apiVersion` | string | yes | `yacd.meigma.io/v1alpha1` | +| `kind` | string | yes | `CardanoNetwork` | +| `metadata` | [ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#objectmeta-v1-meta) | no | Standard object metadata | +| `spec` | [CardanoNetworkSpec](#spec) | yes | Desired state | +| `status` | [CardanoNetworkStatus](#status) | no | Observed state | + +## spec + +`spec` defines the desired state of the network. + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `mode` | string enum | yes | — | Network mode. One of `local`, `public`. | +| `node` | [CardanoNodeSpec](#specnode) | yes | — | Primary cardano-node workload, shared by both modes. | +| `local` | [LocalNetworkSpec](#speclocal) | conditional | — | Generated local devnet. Required when `mode: local`, forbidden otherwise. | +| `public` | [PublicNetworkSpec](#specpublic) | conditional | — | Public-profile join settings. Required when `mode: public`, forbidden otherwise. | +| `chainAPI` | [ChainAPISpec](#specchainapi) | no | — | Network-facing APIs exposed next to the primary node. | + +### Validation rules + +The spec carries CEL validation enforced by the API server: + +- **Mode must match exactly one of `spec.local` or `spec.public`.** When + `mode` is `local`, `local` must be set and `public` must be absent. When + `mode` is `public`, `public` must be set and `local` must be absent. +- **`bootstrap.mithril` is required only when `public.profile` is `mainnet`.** + When the profile is `mainnet`, `public.bootstrap.mithril` must be present. + For any other profile, `public.bootstrap` must be absent. + +### spec.node + +`node` configures the primary cardano-node workload. + +| Field | Type | Required | Default | Constraints | Description | +| --- | --- | --- | --- | --- | --- | +| `version` | string | yes | `11.0.1` | — | cardano-node release used by the managed node image. | +| `image` | string | no | — | — | Full cardano-node image reference. When omitted, the controller derives an image from `version`. | +| `port` | integer (int32) | yes | `3001` | `1`–`65535` | Node-to-node TCP port exposed by the node Service. | +| `storage` | [NodeStorageSpec](#specnodestorage) | no | — | — | Persistent node database storage. | +| `resources` | [ResourceRequirements](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#resourcerequirements-v1-core) | no | — | — | Primary cardano-node container resources. | + +#### spec.node.storage + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `size` | quantity | yes | `10Gi` | Requested persistent volume size for the node database. | +| `storageClassName` | string | no | — | StorageClass used for the node database PVC. | + +### spec.local + +`local` configures a YACD-generated local devnet. Required when `mode: local`. + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `networkMagic` | integer (int64) | yes | `42` | Testnet magic used by local node and client commands. Minimum `0`. | +| `era` | string enum | yes | `conway` | Newest ledger era for the generated network. One of `babbage`, `conway`. | +| `timing` | [LocalNetworkTimingSpec](#speclocaltiming) | yes | — | Slot and epoch duration for the generated network. | +| `topology` | [LocalNetworkTopologySpec](#speclocaltopology) | yes | — | Initial local network topology. | +| `genesis` | [LocalGenesisSpec](#speclocalgenesis) | no | — | Local genesis material generated by YACD. | + +#### spec.local.timing + +| Field | Type | Required | Default | Constraints | Description | +| --- | --- | --- | --- | --- | --- | +| `slotLength` | duration | yes | `100ms` | — | Local network slot duration. | +| `epochLength` | integer (int64) | yes | `500` | minimum `1` | Number of slots in an epoch. | + +#### spec.local.topology + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `pools` | [LocalPoolTopologySpec](#speclocaltopologypools) | yes | — | Generated stake pool nodes. | + +##### spec.local.topology.pools + +| Field | Type | Required | Default | Constraints | Description | +| --- | --- | --- | --- | --- | --- | +| `count` | integer (int32) | yes | `1` | minimum `1` | Number of generated pool nodes. The initial controller may reject values above one until multi-node reconciliation is implemented. | +| `defaults` | [LocalPoolDefaultsSpec](#speclocaltopologypoolsdefaults) | no | — | Shared pool economics for generated pools. | + +###### spec.local.topology.pools.defaults + +| Field | Type | Required | Default | Constraints | Description | +| --- | --- | --- | --- | --- | --- | +| `costLovelace` | integer (int64) | no | — | minimum `0` | Fixed pool cost used for generated pools. | +| `pledgeLovelace` | integer (int64) | no | — | minimum `0` | Pool pledge used for generated pools. | +| `margin` | string | no | — | pattern `^(0(\.[0-9]+)?\|1(\.0+)?)$` | Pool margin as a decimal string in the inclusive range `[0, 1]`. A string to avoid floating-point behavior in the API. | + +#### spec.local.genesis + +| Field | Type | Required | Default | Constraints | Description | +| --- | --- | --- | --- | --- | --- | +| `profile` | string enum | yes | `default` | one of `default`, `zero-fee`, `zero-min-utxo`, `zero-fee-and-min-utxo` | Curated genesis preset. | +| `securityParameter` | integer (int32) | no | — | minimum `1` | Generated Shelley security parameter. | +| `maxLovelaceSupply` | integer (int64) | no | — | minimum `1` | Generated Shelley max lovelace supply. | +| `delegatedSupply` | integer (int64) | no | — | minimum `0` | Lovelace supply initially delegated to generated pools. | +| `protocolVersion` | [ProtocolVersionSpec](#speclocalgenesisprotocolversion) | no | — | — | Initial protocol version in generated genesis material. | + +##### spec.local.genesis.protocolVersion + +| Field | Type | Required | Default | Constraints | Description | +| --- | --- | --- | --- | --- | --- | +| `major` | integer (int32) | yes | — | minimum `0` | Protocol major version. | +| `minor` | integer (int32) | yes | — | minimum `0` | Protocol minor version. | + +The genesis `profile` values control fee and minimum-UTxO behavior: + +| Profile | Effect | +| --- | --- | +| `default` | YACD's normal local development defaults. | +| `zero-fee` | Removes transaction fees for fast local tests. | +| `zero-min-utxo` | Removes the minimum UTxO value. | +| `zero-fee-and-min-utxo` | Removes both fees and the minimum UTxO value. | + +### spec.public + +`public` configures a node that joins a public network profile. Required when +`mode: public`. + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `profile` | string enum | yes | — | Public network profile. One of `preprod`, `preview`, `mainnet`. | +| `bootstrap` | [PublicNetworkBootstrapSpec](#specpublicbootstrap) | conditional | — | Explicit bootstrap behavior. Required (and only allowed) when `profile: mainnet`. | + +!!! warning "Mainnet is opt-in and expensive" + `mainnet` cannot sync from genesis in a development setting. It requires a + [Mithril](https://mithril.network) bootstrap (`public.bootstrap.mithril`) + to seed the node database, large persistent storage, and explicit operator + intent. The CEL rule above forbids `bootstrap` on any non-mainnet profile. + +#### spec.public.bootstrap + +| Field | Type | Required | Default | Description | +| --- | --- | --- | --- | --- | +| `mithril` | [MithrilBootstrapSpec](#specpublicbootstrapmithril) | no | — | Mithril Cardano database bootstrap. Required when `profile: mainnet`. | + +##### spec.public.bootstrap.mithril + +| Field | Type | Required | Default | Constraints | Description | +| --- | --- | --- | --- | --- | --- | +| `image` | string | no | `ghcr.io/input-output-hk/mithril-client:main-2478748` | minLength `1` | Mithril client image used by the bootstrap init container. | +| `snapshot` | string | no | `latest` | minLength `1` | Mithril Cardano database snapshot digest to download. `latest` selects the latest available snapshot. | + +### spec.chainAPI + +`chainAPI` configures APIs exposed next to the primary node. Ogmios and Kupo are +enabled by default. + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `ogmios` | [OgmiosSpec](#specchainapiogmios) | no | Ogmios sidecar and Service. | +| `kupo` | [KupoSpec](#specchainapikupo) | no | Kupo sidecar and Service. | + +#### spec.chainAPI.ogmios + +| Field | Type | Required | Default | Constraints | Description | +| --- | --- | --- | --- | --- | --- | +| `enabled` | boolean | yes | `true` | — | Whether the Ogmios sidecar is deployed. | +| `image` | string | yes | `cardanosolutions/ogmios:v6.14.0` | — | Ogmios image reference. | +| `port` | integer (int32) | yes | `1337` | `1`–`65535` | Ogmios service port. | +| `service` | [ServiceExposureSpec](#serviceexposurespec) | no | — | — | How the Ogmios Service is exposed (ClusterIP or NodePort). | +| `externalURL` | string | no | — | — | Operator-asserted externally reachable URL, mirrored to `status.endpoints.ogmios.externalURL`. See [external access](#external-access). | +| `resources` | [ResourceRequirements](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#resourcerequirements-v1-core) | no | — | — | Ogmios container resources. | + +#### spec.chainAPI.kupo + +| Field | Type | Required | Default | Constraints | Description | +| --- | --- | --- | --- | --- | --- | +| `enabled` | boolean | yes | `true` | — | Whether the Kupo sidecar is deployed. | +| `image` | string | yes | `cardanosolutions/kupo:v2.11.0` | — | Kupo image reference. | +| `port` | integer (int32) | yes | `1442` | `1`–`65535` | Kupo service port. | +| `service` | [ServiceExposureSpec](#serviceexposurespec) | no | — | — | How the Kupo Service is exposed (ClusterIP or NodePort). | +| `externalURL` | string | no | — | — | Operator-asserted externally reachable URL, mirrored to `status.endpoints.kupo.externalURL`. See [external access](#external-access). | +| `resources` | [ResourceRequirements](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#resourcerequirements-v1-core) | no | — | — | Kupo container resources. | + +#### ServiceExposureSpec + +| Field | Type | Required | Default | Constraints | Description | +| --- | --- | --- | --- | --- | --- | +| `type` | string enum | no | `ClusterIP` | `ClusterIP`, `NodePort` | Kubernetes Service type for the endpoint. `NodePort` additionally exposes it on a static node port. | +| `nodePort` | integer (int32) | no | — | `≤ 32767` | Pins the node port when `type` is `NodePort`. `0` or omitted auto-assigns from the node-port range (30000–32767); a pinned value is valid only with `NodePort`. | + +#### External access + +`service` and `externalURL` let a network advertise a directly reachable address so +the CLI can skip the per-command port-forward (see +[Connecting tools](../developer/connecting-tools.md)). They are an +**advertisement**: the operator publishes `externalURL`, but making it routable is +the provisioner's job — `NodePort` plus a host-port mapping for `yacd devnet`, or an +ingress for a shared cluster. `externalURL` is validated leniently (an absolute URL +with a host and a `ws`, `wss`, `http`, or `https` scheme) because the CLI probes it +for reachability before trusting it. The in-cluster `url` in status is unchanged; +`externalURL` is an additive peer. + +!!! note "No faucet field — funding is CLI-native" + There is no `chainAPI.faucet` or `chainAPI.wallet` field. Every local network + is created with a genesis-funded `faucet` wallet, stored as a + `-wallet-faucet` Secret; fund developer wallets from it with the + [`yacd wallet`](cli.md#wallet) verbs. The faucet wallet has no `spec` knobs + and no `status` block. + +## status + +`status` reports the observed state of the network. All fields are populated by +the controller and are read-only. + +| Field | Type | Description | +| --- | --- | --- | +| `observedGeneration` | integer (int64) | Most recent generation observed by the controller. | +| `network` | [CardanoNetworkIdentityStatus](#statusnetwork) | Resolved network identity once chain material is generated or loaded. | +| `endpoints` | [CardanoNetworkEndpointsStatus](#statusendpoints) | Cluster-local connection details for clients and supporting controllers. | +| `sync` | [CardanoNetworkSyncStatus](#statussync) | Primary node chain synchronization status inferred from in-cluster sources. | +| `conditions` | [][Condition](#conditions) | Current state of the resource (list-map keyed by `type`). | + +### status.network + +| Field | Type | Description | +| --- | --- | --- | +| `mode` | string enum | Resolved network mode. One of `local`, `public`. | +| `localnetFingerprint` | string | Accepted fingerprint for generated localnet inputs. Changing those inputs requires deleting and recreating the CardanoNetwork. | +| `networkFingerprint` | string | Accepted fingerprint for the resolved network profile, independent of local/public. Changing it requires deleting and recreating the CardanoNetwork. | +| `networkMagic` | integer (int64) | Resolved Cardano network magic. | +| `profile` | string enum | Resolved public network profile, when `mode` is `public`. One of `preprod`, `preview`, `mainnet`. | +| `era` | string enum | Newest resolved ledger era known to the controller. One of `babbage`, `conway`. | + +### status.endpoints + +Each endpoint is a [ServiceEndpointStatus](#serviceendpointstatus). + +| Field | Type | Description | +| --- | --- | --- | +| `nodeToNode` | ServiceEndpointStatus | Primary node-to-node endpoint. | +| `ogmios` | ServiceEndpointStatus | Ogmios JSON/RPC endpoint. | +| `kupo` | ServiceEndpointStatus | Kupo chain index HTTP endpoint. | +| `artifacts` | ServiceEndpointStatus | cardano-tools `serve` HTTP endpoint exposing the staged network artifact files and `manifest.json`. | + +#### ServiceEndpointStatus + +| Field | Type | Description | +| --- | --- | --- | +| `serviceName` | string | Kubernetes Service name. | +| `port` | integer (int32) | Service port. | +| `url` | string | In-cluster convenience URL for protocols with a stable URL shape. | +| `externalURL` | string | Operator-asserted externally reachable URL, mirrored from the spec. Empty unless the network sets `chainAPI..externalURL`. | + +### status.sync + +`sync` reports the primary node's observed synchronization state using only +in-cluster sources. + +| Field | Type | Constraints | Description | +| --- | --- | --- | --- | +| `source` | string enum | one of `ogmios` | Probe source that produced this sync status. | +| `connectionStatus` | string | — | Ogmios health connection status. | +| `tip` | [CardanoNetworkTipStatus](#statussynctip) | — | Last known chain tip reported by Ogmios. | +| `lastTipUpdate` | timestamp (date-time) | — | Timestamp Ogmios reported for the last known tip update. | +| `observedAt` | timestamp (date-time) | — | Controller time when this sync status was probed. | +| `networkSynchronization` | number (float64) | `0`–`1` | Ogmios' synchronization estimate, rounded to five decimals when reported. | +| `inferredTipSlot` | integer (int64) | minimum `0` | Wall-clock network slot inferred from the published Shelley genesis timing. | +| `lagSlots` | integer (int64) | minimum `0` | `max(inferredTipSlot - tip.slot, 0)`. | +| `lagSeconds` | integer (int64) | minimum `0` | `lagSlots` converted through `slotLength` and rounded up to a whole number of seconds. | + +#### status.sync.tip + +| Field | Type | Constraints | Description | +| --- | --- | --- | --- | +| `slot` | integer (int64) | minimum `0` | Tip slot reported by Ogmios. | +| `blockHeight` | integer (int64) | minimum `0` | Tip block height reported by Ogmios. | +| `hash` | string | — | Tip hash reported by Ogmios. | + +### conditions + +`conditions` is a list of standard +[metav1.Condition](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.30/#condition-v1-meta) +entries, keyed by a unique `type`. Each condition `status` is one of `True`, +`False`, or `Unknown`. + +| Type | Meaning | +| --- | --- | +| `Ready` | The network is usable through its published endpoints. | +| `DBSyncAttachmentReady` | The primary-sidecar db-sync attachment is not blocking the primary Pod. | +| `NodeReady` | The primary node container is running. | +| `NodeSynchronized` | The primary node is caught up to its inferred network tip. | +| `NodeProgressing` | The primary node tip is advancing or already synchronized. | +| `ArtifactsReady` | The network artifact bundle is staged and served over HTTP. | +| `OgmiosReady` | Ogmios is enabled and connected to the primary node. | +| `KupoReady` | Kupo is enabled and synchronized enough to serve its API. | +| `Progressing` | The resource is being created or updated. | +| `Degraded` | The resource failed to reach or maintain its desired state. | diff --git a/docs/reference/cli.md b/docs/reference/cli.md new file mode 100644 index 0000000..b99ac00 --- /dev/null +++ b/docs/reference/cli.md @@ -0,0 +1,582 @@ +# CLI reference + +The `yacd` developer CLI manages YACD environments in a Kubernetes cluster and +wires local tools and tests to a running network. This page is the single source +of truth for every command, flag, default, and the `YACD_*` environment +contract. Other pages link here rather than restating flags. + +## Synopsis + +```text +yacd [command] [flags] +``` + +Each command that targets a network takes a positional `NAME`. `NAME` becomes +the CardanoNetwork name, and the namespace defaults to `NAME` unless you pass +`--namespace`. Both `NAME` and the resolved namespace must be valid DNS-1123 +labels (lowercase alphanumeric and `-`); invalid input is rejected. + +Commands: + +| Command | Summary | +| --- | --- | +| `devnet` | Bring up a local Cardano devnet (cluster, operator, and a funded network). | +| `devnet down` | Delete the managed devnet cluster. | +| `devnet status` | Show the managed devnet cluster, operator, and network status. | +| `install` | Install or upgrade the YACD operator on a cluster. | +| `init` | Print a commented `yacd.yaml` environment template to stdout. | +| `up NAME` | Create or update a YACD environment and wait for readiness. | +| `down NAME` | Delete a YACD environment and wait for clean removal. | +| `list` | List YACD environments across all namespaces (or one with `-n`). | +| `info NAME` | Print CardanoNetwork status and connection information. | +| `wallet NET` | Manage developer wallets: add, list, topup, export, remove. | +| `run NAME [-- command ...]` | Run a command (or a shell) on the host with the `YACD_*` environment wired to forwarded endpoints. | +| `connect NAME` | Forward a network's endpoints and hold them open until interrupted. | +| `exec NAME -- command ...` | Run a command inside the primary node Pod (for socket-bound tools). | +| `completion` | Generate a shell autocompletion script. | + +## Global flags + +These persistent flags apply to every command. Each binds to a `YACD_*` +environment variable through the `YACD` env prefix; precedence is **flag > env > +default**. + +| Flag | Short | Type | Default | Env override | Meaning | +| --- | --- | --- | --- | --- | --- | +| `--kubeconfig` | | string | `""` | `YACD_KUBECONFIG` | Path to the kubeconfig file. Empty defers to standard loading rules. | +| `--context` | | string | `""` | `YACD_CONTEXT` | Kubeconfig context to use. Empty defers to the current context. | +| `--namespace` | `-n` | string | `""` | `YACD_NAMESPACE` | Kubernetes namespace. Empty defers to `NAME` (per-environment namespace) or kubeconfig defaults. | +| `--log-level` | | string | `info` | `YACD_LOG_LEVEL` | Log level: `debug`, `info`, `warn`, `error`. | +| `--log-format` | | string | `text` | `YACD_LOG_FORMAT` | Log format: `text`, `json`. | +| `--help` | `-h` | bool | `false` | | Help for the command. | +| `--version` | `-v` | bool | `false` | | Print the version. Root command only. | + +!!! note "Env override naming" + The env prefix is `YACD` and flag names are upper-cased with `-` replaced by + `_`. `YACD_NAMESPACE` is both the override for `--namespace` and a value the + CLI publishes into the `YACD_*` contract; see [the contract table](#the-yacd-environment-contract). + +`yacd --version` prints `yacd () built `. + +## devnet + +```text +yacd devnet [flags] +``` + +Brings a managed [k3d](https://k3d.io) cluster, the operator, and a default +funded local network to a ready state in one command. Takes no `NAME` and no +`--namespace` (it manages a fixed `devnet` network in a `devnet` namespace). + +`devnet` requires a running [Docker](https://www.docker.com/) (or compatible +container runtime). It does not require a preinstalled k3d: on first use the CLI +downloads a version-pinned, SHA256-verified k3d binary and caches it under +`$XDG_DATA_HOME/yacd/bin` (default `~/.local/share/yacd/bin`). + +| Flag | Type | Default | Meaning | +| --- | --- | --- | --- | +| `--bare` | bool | `false` | Stop after installing the operator; apply no network. | +| `--timeout` | duration | `12m0s` | Maximum time to wait for the cluster, operator, and network. Must be greater than 0. | + +On success it prints the cluster context, the operator version, and (unless +`--bare`) the Ogmios and Kupo endpoints, the funded wallet address, and a +copy-pasteable `yacd exec` tip-query hint. + +`devnet` exposes Ogmios and Kupo as NodePort Services mapped to fixed host ports, +so they are reachable from your machine at `ws://localhost:1337` (Ogmios) and +`http://localhost:1442` (Kupo) without a port-forward. The printed endpoints +advertise those host URLs, and the CLI uses them automatically (see +[chain access resolution](#chain-access-resolution)). If port 1337 or 1442 is +already in use, `devnet` fails with a clear collision error. + +### devnet down + +```text +yacd devnet down [flags] +``` + +Deletes the managed devnet cluster and restores the prior kubectl context. + +| Flag | Type | Default | Meaning | +| --- | --- | --- | --- | +| `--timeout` | duration | `5m0s` | Maximum time to wait for the cluster to be deleted. Must be greater than 0. | + +### devnet status + +```text +yacd devnet status +``` + +Read-only unified view of the managed cluster, operator, and networks. Takes no +flags beyond the [global flags](#global-flags). Prints a one-line hint when no +managed cluster exists. + +## install + +```text +yacd install [flags] +``` + +Installs or upgrades the YACD operator on the targeted cluster, then waits for +the manager to become ready. The target is the explicit `--kubeconfig`/`--context` +(or the `YACD_KUBECONFIG`/`YACD_KUBE_CONTEXT` variables), otherwise the ambient +current-context; `install` never targets the managed devnet. The global +`-n`/`--namespace` flag selects the install namespace and defaults to +`yacd-system` (created if absent). + +`install` reconciles the cluster to the operator version this CLI embeds: it +installs when absent, upgrades an older same-major install, re-applies an equal +version to heal drift, and refuses a newer or major-mismatched in-cluster version +with actionable guidance. The operator image is pinned to the chart's appVersion +(the version this CLI embeds); the supported way to change the operator version +is to upgrade the CLI. + +| Flag | Short | Type | Default | Meaning | +| --- | --- | --- | --- | --- | +| `--wait` | | bool | `true` | Wait for the manager Deployment to become Available. | +| `--timeout` | | duration | `5m0s` | Maximum time to wait for readiness (also bounds a `--dry-run` plan's reads). Must be greater than 0 when `--wait` is set. | +| `--dry-run` | | bool | `false` | Report the planned action without changing the cluster. | +| `--values` | `-f` | stringArray | `[]` | Path to a YAML file of operational chart value overrides (repeatable; later files win). | +| `--set` | | stringArray | `[]` | Set an operational chart value (Helm `--set` syntax, repeatable). | +| `--set-string` | | stringArray | `[]` | Set an operational chart value forced to a string (repeatable). | + +The override flags customize **operational** chart values (replicas, resources, +scheduling, logging, metrics, and so on), validated against the chart's schema so +a bad value fails fast (under `--dry-run` too). Precedence, later wins: `-f` files +(in order) < `--set` < `--set-string`. The operator `image.*` values are not part +of the supported surface: a `--set image.tag`, `image.repository`, or +`image.digest` will repoint the operator image, but the supported way to change +the operator version is to upgrade the CLI. + +`--dry-run` prints the action the next install would take and changes nothing: + +```text +Plan: install operator (installed none -> v0.2.1) in namespace yacd-system +``` + +See [Installation](../operator/installation.md) for the full operator-install +workflow (including the Helm alternative), and the +[configuration reference](configuration.md) for every value the override flags +accept. + +## init + +```text +yacd init +``` + +Prints a fully-commented developer environment template to stdout and takes no +arguments or flags beyond the [global flags](#global-flags). The active +configuration is a ready-to-run local devnet; local networks automatically get a +genesis-funded `faucet` wallet. Commented blocks document the rest of the API, +including chain-API overrides and a public/mainnet alternative. Redirect it to a +file and apply it: + +```sh +yacd init > yacd.yaml +yacd up dev -f yacd.yaml +``` + +See [Defining networks](../developer/networks.md) for the scaffold-and-edit +workflow and the [Environment file reference](environment.md) for field details. + +## up + +```text +yacd up NAME [flags] +``` + +Loads a developer environment file, renders it into a CardanoNetwork under the +resolved identity, creates the namespace if needed, and server-side-applies the +network. Unless `--wait=false`, it then polls until the network is Ready or the +timeout elapses. + +| Flag | Short | Type | Default | Meaning | +| --- | --- | --- | --- | --- | +| `--file` | `-f` | string | `""` | Developer environment file. Required. | +| `--dry-run` | | bool | `false` | Render the manifest to stdout without applying it. | +| `--allow-mainnet` | | bool | `false` | Allow applying a mainnet CardanoNetwork. | +| `--wait` | | bool | `true` | Wait for the CardanoNetwork to become ready. | +| `--timeout` | | duration | `12m0s` | Maximum time to wait for readiness. Must be greater than 0 when `--wait` is set. | + +!!! warning "Mainnet requires `--allow-mainnet`" + Applying a mainnet network without `--allow-mainnet` is rejected because + mainnet deployments create large persistent volumes and bootstrap from + [Mithril](https://mithril.network). `--dry-run` renders a mainnet manifest + without the flag but applies nothing. + +## down + +```text +yacd down NAME [flags] +``` + +Deletes the named CardanoNetwork and, unless `--wait=false`, blocks until the +object and its garbage-collected children are gone. Deletion is idempotent: a +network that is already absent is reported as success. + +| Flag | Type | Default | Meaning | +| --- | --- | --- | --- | +| `--wait` | bool | `true` | Wait for the CardanoNetwork and its resources to be removed. | +| `--timeout` | duration | `5m0s` | Maximum time to wait for removal. Must be greater than 0 when `--wait` is set. | + +## list + +```text +yacd list [flags] +``` + +Lists CardanoNetworks across all namespaces by default, projecting each into +`name`, `namespace`, `mode`, `ready`, and published `endpoints`. Scope to a +single namespace with the global `-n`/`--namespace` flag. Renders an aligned +table by default or JSON with `--json`. + +| Flag | Type | Default | Meaning | +| --- | --- | --- | --- | +| `--json` | bool | `false` | Print machine-readable JSON. | + +The table columns are `NAME`, `NAMESPACE`, `MODE`, `READY`, `ENDPOINTS`. +`READY` reflects a fresh `Ready` condition observed as `True` (a stale status is +reported as not ready). `ENDPOINTS` is a comma-separated list of published +endpoint names (`node-to-node`, `ogmios`, `kupo`) or `-` when none are published +yet. + +The `--json` output is an array of objects with fields: + +| Field | Type | Meaning | +| --- | --- | --- | +| `name` | string | CardanoNetwork name. | +| `namespace` | string | CardanoNetwork namespace. | +| `mode` | string | Requested network mode (`local` or `public`). | +| `ready` | bool | Fresh `Ready` condition observed as `True`. | +| `endpoints` | object | `nodeToNode`, `ogmios`, `kupo` URLs; empty when unpublished. | + +## info + +```text +yacd info NAME [flags] +``` + +Fetches the named CardanoNetwork and prints its status and connection +information, as human-readable text by default or JSON with `--json`. + +| Flag | Type | Default | Meaning | +| --- | --- | --- | --- | +| `--json` | bool | `false` | Print machine-readable JSON. | + +The `--json` object has stable field names. The `conditions` array is always +present (possibly empty); nil sub-statuses are omitted rather than emitted as +empty objects. + +| Field | Type | Meaning | +| --- | --- | --- | +| `name` | string | CardanoNetwork name. | +| `namespace` | string | CardanoNetwork namespace. | +| `observedGeneration` | int | Last generation the controller observed. Omitted when 0. | +| `network` | object | `mode`, `localnetFingerprint`, `networkMagic`, `profile`, `era`. | +| `endpoints` | object | `nodeToNode`, `ogmios`, `kupo`, each `{serviceName, port, url}` or absent. | +| `wallet` | object | `{address, keySecretName}` of the genesis-funded `faucet` wallet. Omitted when the network has none (non-local networks). | +| `conditions` | array | Each `{type, status, reason, message, observedGeneration, lastTransitionTime}` (RFC3339 timestamp). | + +## wallet + +```text +yacd wallet NET [args] [flags] +``` + +Manages developer wallets for a network and funds them by building, signing, and +submitting transactions directly over the network's Ogmios and Kupo endpoints; +there is no in-cluster faucet service. Keys are stored as labeled Kubernetes +Secrets (`-wallet-`) in the network's namespace, and the CLI reads +them to sign locally. + +Every local network has a reserved, operator-owned genesis-funded `faucet` wallet +that funding spends from by default. The `WALLET` argument of `topup`, `remove`, +and `export` accepts a managed wallet name, a public key (hex), or a bech32 +`addr_test...` address. + +### wallet list + +```text +yacd wallet list NET [flags] +``` + +Lists the CLI-managed wallets for a network (name, address, and the +`managed-by-cli` source); the operator-owned `faucet` wallet is not listed. +`--json` prints a machine-readable array. + +### wallet add + +```text +yacd wallet add NET [flags] +``` + +Generates a new managed wallet and, with `--topup`, funds it from the `faucet` +wallet. + +| Flag | Type | Default | Meaning | +| --- | --- | --- | --- | +| `--name` | string | generated | Wallet name (default: a generated adjective-noun name). | +| `--topup` | string | `""` | Fund the new wallet with this many lovelace from the faucet. | +| `--await` | bool | `false` | Wait for the funding transaction to confirm on-chain (requires `--topup`). | +| `--await-timeout` | duration | `2m0s` | Maximum time to wait for `--await` confirmation. | +| `--ogmios-url` | string | `""` | Ogmios URL to use for funding; overrides the [resolved endpoint](#chain-access-resolution). | +| `--kupo-url` | string | `""` | Kupo URL to use for funding; overrides the [resolved endpoint](#chain-access-resolution). | +| `--json` | bool | `false` | Print machine-readable JSON. | + +### wallet topup + +```text +yacd wallet topup NET WALLET LOVELACE [flags] +``` + +Funds `WALLET` with `LOVELACE` (positional, must be greater than 0) from the +`faucet` wallet, or from another managed wallet with `--from`. It reaches Ogmios +and Kupo on its own (see [chain access resolution](#chain-access-resolution)), so +no `yacd run` wrapper is needed. + +| Flag | Type | Default | Meaning | +| --- | --- | --- | --- | +| `--from` | string | `faucet` | Source wallet name to fund from (default: the faucet wallet). | +| `--await` | bool | `false` | Wait for the funding transaction to confirm on-chain. | +| `--await-timeout` | duration | `2m0s` | Maximum time to wait for `--await` confirmation. | +| `--ogmios-url` | string | `""` | Ogmios URL to use for funding; overrides the [resolved endpoint](#chain-access-resolution). | +| `--kupo-url` | string | `""` | Kupo URL to use for funding; overrides the [resolved endpoint](#chain-access-resolution). | +| `--json` | bool | `false` | Print machine-readable JSON. | + +### wallet export + +```text +yacd wallet export NET WALLET [flags] +``` + +Writes the wallet's `.skey`, `.vkey`, and `.addr` files +(mode `0600`) to `.yacd///wallets//`. + +| Flag | Type | Default | Meaning | +| --- | --- | --- | --- | +| `--out` | string | `.yacd///wallets/` | Directory to write the wallet files into. | +| `--force` | bool | `false` | Overwrite existing wallet files. | + +### wallet remove + +```text +yacd wallet remove NET WALLET +``` + +Deletes a managed wallet. The reserved `faucet` wallet cannot be removed. + +## Chain access resolution + +`yacd run` and the wallet funding commands resolve a host-usable URL for each of +Ogmios and Kupo, preferring a directly reachable endpoint over an ephemeral +port-forward. Per endpoint, the first available rung wins: + +1. An explicit `--ogmios-url` / `--kupo-url` flag (funding commands only). +2. An ambient `YACD_OGMIOS_URL` / `YACD_KUPO_URL` (set inside `yacd run`). +3. The network's `status.endpoints..externalURL`, when a short + reachability probe (a ~2s TCP dial) succeeds. +4. An ephemeral `kubectl` port-forward — the universal fallback. + +So a network that advertises a reachable `externalURL` — `yacd devnet` pins one on +`localhost`, and a shared cluster can front one with an ingress — is reached +without a port-forward. `yacd connect` always forwards; it is the remote-access +tool. + +## run + +```text +yacd run NAME [-- command [args...]] [flags] +``` + +Resolves a host-usable URL for each chain-API endpoint (see +[chain access resolution](#chain-access-resolution)) — using a reachable +`externalURL` when one exists and otherwise a scoped port-forward — injects the +[`YACD_*` environment](#the-yacd-environment-contract), and execs the command (or +your `$SHELL`, falling back to `/bin/sh`, when none is given) on the host with +that environment. Any forwards are torn down when the command exits. + +Put `--` before any command that takes its own flags so they are passed through +to the command instead of being parsed by `yacd`. + +```sh +# Run a test suite against the network (note the -- before the command) +yacd run my-net -- go test ./e2e/... + +# Open a shell with the YACD_* environment set +yacd run my-net +``` + +`run` has no command-specific flags beyond the [global flags](#global-flags). +The child inherits the CLI's stdio and process group, so an interactive Ctrl-C +reaches it. The child's exit status is propagated to your shell; a process +killed by a signal reports `128+signal` (for example `130` for SIGINT). If the +forwards drop while the command runs, `run` exits non-zero and reports the lost +connection. + +## connect + +```text +yacd connect NAME [flags] +``` + +Establishes supervised port-forwards to a network's chain-API endpoints, writes +the loopback URLs to `.yacd//endpoints.json` (or +`.yacd///endpoints.json` when `--namespace` is set), prints +them, and holds them open until interrupted (Ctrl-C). Run it in one terminal and +your tools in another. Dropped forwards are re-established automatically (with a +freshly resolved Pod and new local ports). On exit (or before each +re-establish), the endpoints file is removed. + +`connect` has no command-specific flags beyond the [global flags](#global-flags). + +!!! note "Loopback ports are ephemeral" + The endpoints file holds no secrets, and its ports are only live while + `connect` is running. See the [endpoints.json schema](#the-endpointsjson-schema). + +## exec + +```text +yacd exec NAME -- command [args...] [flags] +``` + +Runs a command inside the primary `cardano-node` Pod with kubectl-exec +semantics, for tools that reach the node over its local Unix socket (notably +`cardano-cli`) rather than over a forwarded TCP port. `CARDANO_NODE_SOCKET_PATH` +and the [`YACD_*` variables](#the-yacd-environment-contract) are set in the pod +environment, so `cardano-cli` finds the socket automatically. A command is +required. + +```sh +# cardano-cli reads CARDANO_NODE_SOCKET_PATH from the pod environment: +yacd exec my-net -- cardano-cli query tip --testnet-magic 42 + +# To interpolate YACD_* variables into arguments, run a shell explicitly: +yacd exec my-net -- sh -c 'cardano-cli query tip --testnet-magic "$YACD_NETWORK_MAGIC"' + +# From a terminal, open an interactive shell in the node Pod: +yacd exec my-net -- sh +``` + +`exec` has no command-specific flags beyond the [global flags](#global-flags). +The command is run directly, not through a shell, so `$VAR` references in +arguments are **not** expanded; wrap the command in `sh -c '...'` to interpolate +`YACD_*` variables. When both stdin and stdout are terminals, `exec` attaches an +interactive TTY; piped or non-terminal (CI) invocations stream without one. The +command's exit code is propagated to the caller. `exec` requires the +CardanoNetwork to be Ready. + +## The YACD environment contract + +`run`, `connect`, and `exec` publish a stable, versioned set of `YACD_*` +variables (contract version 1). Tests and tooling read these instead of parsing +any YACD file. The variable names are identical whether a command runs on the +host (`run`, over the resolved chain access) or inside the primary Pod (`exec`, +over cluster DNS); only the values adapt. Adding a variable is backward +compatible; renaming or removing one is a breaking change to the contract. + +`YACD_OGMIOS_URL` and `YACD_KUPO_URL` are also **inputs**: the funding commands +and a nested `yacd run` read them during +[chain access resolution](#chain-access-resolution), so a tool launched inside +`yacd run` reuses its already-resolved endpoints instead of opening its own. + +| Variable | Host (`run`) value | In-pod (`exec`) value | Present when | +| --- | --- | --- | --- | +| `YACD_NETWORK` | network name | network name | always | +| `YACD_NAMESPACE` | network namespace | network namespace | always | +| `YACD_NETWORK_MAGIC` | network magic (integer) | network magic (integer) | when the controller has published the network magic | +| `YACD_OGMIOS_URL` | loopback URL (scheme preserved, e.g. `ws://`) | published ClusterIP URL | when Ogmios is published and forwarded | +| `YACD_KUPO_URL` | loopback URL | published ClusterIP URL | when Kupo is published and forwarded | +| `CARDANO_NODE_SOCKET_PATH` | *not set* | `/ipc/node.socket` | in-pod `exec` only | + +Notes: + +- On the host, each forwarded chain endpoint URL is rewritten onto + `127.0.0.1:`. Only host and port change; the scheme, path, query, + and fragment carry through unchanged (a `ws://` Ogmios endpoint stays `ws://`). +- In the Pod, the published ClusterIP URLs are passed through verbatim. +- The node-to-node endpoint is intentionally excluded: it is a TCP peer + protocol, not something host or in-pod test tooling speaks. +- `CARDANO_NODE_SOCKET_PATH` is unprefixed because that is the name + `cardano-cli` already expects. + +`yacd wallet topup --await` forwards Ogmios and Kupo itself, so it confirms +on-chain without needing these variables; inside `yacd run` it reuses the +forwards already established. + +## The endpoints.json schema + +`yacd connect` writes a token-free connection document to +`.yacd//endpoints.json` (mode `0600`, in a `0700` directory), or +`.yacd///endpoints.json` when `--namespace` differs from +`NAME`. The field names are stable across releases. + +```json +{ + "network": "my-net", + "namespace": "my-net", + "networkMagic": 42, + "ogmiosUrl": "ws://127.0.0.1:51820", + "kupoUrl": "http://127.0.0.1:51821" +} +``` + +| Field | Type | Meaning | +| --- | --- | --- | +| `network` | string | CardanoNetwork name. Always present. | +| `namespace` | string | CardanoNetwork namespace. Always present. | +| `networkMagic` | int | Network magic. Omitted until the controller publishes it. | +| `ogmiosUrl` | string | Loopback Ogmios URL. Omitted when not forwarded. | +| `kupoUrl` | string | Loopback Kupo URL. Omitted when not forwarded. | + +The ports it lists are only live while `connect` is running, and the file is +removed on disconnect. + +## Install + +Download the `yacd` binary for your platform from the +[releases page](https://github.com/meigma/yacd/releases) and place it on your +`PATH`, then verify: + +```sh +yacd --version +``` + +## Shell completion + +`yacd completion ` generates an autocompletion script. Supported shells +are `bash`, `zsh`, `fish`, and `powershell`. Each script accepts +`--no-descriptions` to disable completion descriptions. + +=== "macOS" + + ```sh + # zsh + yacd completion zsh > $(brew --prefix)/share/zsh/site-functions/_yacd + + # bash (requires the bash-completion package) + yacd completion bash > $(brew --prefix)/etc/bash_completion.d/yacd + ``` + +=== "Linux" + + ```sh + # zsh + yacd completion zsh > "${fpath[1]}/_yacd" + + # bash (requires the bash-completion package) + yacd completion bash > /etc/bash_completion.d/yacd + ``` + +Start a new shell for the setup to take effect. To load completions for the +current session only: + +```sh +source <(yacd completion zsh) # or bash, fish +``` + +For `zsh`, if completion is not yet enabled, run once: + +```sh +echo "autoload -U compinit; compinit" >> ~/.zshrc +``` diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md new file mode 100644 index 0000000..eaa2e3d --- /dev/null +++ b/docs/reference/configuration.md @@ -0,0 +1,183 @@ +# Configuration + +Reference for configuring the YACD operator. There are two layers: + +1. **Helm chart values** — what you set in `values.yaml` (or `--set`) when you + install the operator. Both `helm install` and `yacd install` consume these + values; `yacd install` takes them via `-f`/`--set`/`--set-string` and + validates them against the chart schema. They render the operator Deployment, + RBAC, metrics Service, and the optional Kyverno image-verification policy. +2. **Manager flags** — the command-line flags the manager process accepts. The + chart translates the relevant values into these flags automatically; set them + directly only when running the manager binary outside the chart. + +!!! note "Operator image with `yacd install`" + With `yacd install`, the operator image is pinned to the chart's `appVersion` + (the version the CLI embeds). Changing `image.*` is not the supported way to + move the operator version — upgrade the CLI instead. The other operational + values behave the same across both install methods. + +This page documents the operator's own configuration. For the `yacd` CLI flags, +environment variables, and `endpoints.json` schema, see +[CLI reference](cli.md). For the per-network and per-db-sync resource fields, see +[CardanoNetwork reference](cardanonetwork.md) and +[CardanoDBSync reference](cardanodbsync.md). + +## Helm chart values + +Defaults below are from `charts/yacd/values.yaml`. Only user-relevant keys are +listed. + +### Images + +| Key | Default | Meaning | +| --- | --- | --- | +| `image.repository` | `ghcr.io/meigma/yacd` | Manager image repository. | +| `image.tag` | `""` | Manager image tag. Empty uses the chart `appVersion`. | +| `image.digest` | `""` | Manager image digest (`sha256:...`). When set, it takes precedence over `image.tag`. | +| `image.pullPolicy` | `IfNotPresent` | Manager image pull policy. | +| `cardanoTestnet.image.repository` | `""` | Overrides the cardano-testnet tools image (the create-env init container and the primary cardano-node container when `spec.node.image` is unset). Empty uses the operator's built-in versioned reference. | +| `cardanoTestnet.image.tag` | `""` | Tag for the cardano-testnet override. Applied only when `repository` is set. | +| `cardanoTestnet.image.digest` | `""` | Digest for the cardano-testnet override. When set, it takes precedence over the tag. | +| `cardanoTools.image.repository` | `""` | Overrides the cardano-tools utility image used for artifact staging containers. Empty uses the operator's built-in versioned reference. | +| `cardanoTools.image.tag` | `""` | Tag for the cardano-tools override. Applied only when `repository` is set. | +| `cardanoTools.image.digest` | `""` | Digest for the cardano-tools override. When set, it takes precedence over the tag. | +| `imagePullSecrets` | `[]` | Image pull secrets added to the manager pod. | + +The `cardanoTestnet` and `cardanoTools` overrides only render a flag when +`repository` is set; otherwise the operator keeps its built-in +`:-yacd.N` reference. These exist to run pre-release publisher +changes that the published tags do not yet contain. + +### Manager runtime + +| Key | Default | Meaning | +| --- | --- | --- | +| `replicaCount` | `1` | Number of manager replicas. | +| `manager.logFormat` | `json` | Log output format. Rendered into `--log-format`. One of `json`, `text`. | +| `manager.logLevel` | `info` | Minimum log level. Rendered into `--log-level`. One of `debug`, `info`, `warn`, `error`. | +| `manager.enableHTTP2` | `false` | Enable HTTP/2 for the metrics and webhook servers. Rendered into `--enable-http2` when `true`. | +| `manager.healthProbe.port` | `8081` | Health/readiness probe port. Rendered into `--health-probe-bind-address=:`. | +| `manager.extraArgs` | `[]` | Extra raw arguments appended verbatim to the manager command line. | +| `leaderElection.enabled` | `true` | Enable controller-runtime leader election. Renders `--leader-elect` when `true`. | + +!!! note + The chart enables leader election by default (`leaderElection.enabled: true`), + while the manager binary's `--leader-elect` flag defaults to `false`. The + chart adds the flag for you. + +### Metrics and TLS + +| Key | Default | Meaning | +| --- | --- | --- | +| `metrics.enabled` | `true` | Serve the metrics endpoint. When `false`, the chart renders `--metrics-bind-address=0` to disable it. | +| `metrics.port` | `8443` | Metrics container port. Rendered into `--metrics-bind-address=:`. | +| `metrics.secure` | `true` | Serve metrics over HTTPS with Kubernetes authn/authz. Rendered into `--metrics-secure=`. | +| `metrics.service.create` | `true` | Create the metrics Service. | +| `metrics.service.port` | `8443` | Metrics Service port. | +| `metrics.service.annotations` | `{}` | Annotations on the metrics Service. | +| `metrics.tls.secretName` | `""` | Secret holding metrics server TLS material. When set, the chart mounts it and renders the `--metrics-cert-*` flags. | +| `metrics.tls.certPath` | `/certs/metrics` | Mount path for the metrics TLS Secret. Rendered into `--metrics-cert-path` when `secretName` is set. | +| `metrics.tls.certName` | `tls.crt` | Certificate filename. Rendered into `--metrics-cert-name`. | +| `metrics.tls.keyName` | `tls.key` | Private key filename. Rendered into `--metrics-cert-key`. | +| `webhook.tls.secretName` | `""` | Secret holding webhook server TLS material. When set, the chart mounts it and renders the `--webhook-cert-*` flags. | +| `webhook.tls.certPath` | `/certs/webhook` | Mount path for the webhook TLS Secret. Rendered into `--webhook-cert-path` when `secretName` is set. | +| `webhook.tls.certName` | `tls.crt` | Certificate filename. Rendered into `--webhook-cert-name`. | +| `webhook.tls.keyName` | `tls.key` | Private key filename. Rendered into `--webhook-cert-key`. | + +### RBAC and service account + +| Key | Default | Meaning | +| --- | --- | --- | +| `serviceAccount.create` | `true` | Create the manager ServiceAccount. | +| `serviceAccount.name` | `""` | ServiceAccount name. Empty derives the name from the release. | +| `serviceAccount.annotations` | `{}` | Annotations on the ServiceAccount. | +| `rbac.create` | `true` | Create the manager Role/RoleBinding and metrics auth RBAC. | +| `rbac.metricsReader.create` | `true` | Create the metrics-reader ClusterRole for scraping the protected metrics endpoint. | + +### Scheduling and resources + +| Key | Default | Meaning | +| --- | --- | --- | +| `resources.requests.cpu` | `10m` | Manager CPU request. (No limits set by default.) | +| `resources.requests.memory` | `64Mi` | Manager memory request. | +| `nodeSelector` | `{}` | Manager pod node selector. | +| `tolerations` | `[]` | Manager pod tolerations. | +| `affinity` | `{}` | Manager pod affinity. | +| `topologySpreadConstraints` | `[]` | Manager pod topology spread constraints. | +| `extraVolumes` | `[]` | Extra volumes added to the manager pod. | +| `extraVolumeMounts` | `[]` | Extra volume mounts added to the manager container. | + +### Security contexts + +| Key | Default | Meaning | +| --- | --- | --- | +| `podSecurityContext.runAsNonRoot` | `true` | Require the pod to run as a non-root user. | +| `podSecurityContext.seccompProfile.type` | `RuntimeDefault` | Pod seccomp profile. | +| `containerSecurityContext.allowPrivilegeEscalation` | `false` | Disallow privilege escalation. | +| `containerSecurityContext.capabilities.drop` | `[ALL]` | Linux capabilities dropped. | +| `containerSecurityContext.readOnlyRootFilesystem` | `true` | Mount the root filesystem read-only. | +| `containerSecurityContext.runAsNonRoot` | `true` | Require the container to run as a non-root user. | +| `containerSecurityContext.runAsUser` | `65532` | Container UID. | + +### Labels and annotations + +| Key | Default | Meaning | +| --- | --- | --- | +| `nameOverride` | `""` | Override the chart name component of generated names. | +| `fullnameOverride` | `""` | Override the full resource name prefix. | +| `commonLabels` | `{}` | Labels added to all chart resources. | +| `commonAnnotations` | `{}` | Annotations added to all chart resources. | +| `deploymentAnnotations` | `{}` | Annotations on the manager Deployment. | +| `podLabels` | `{}` | Labels on the manager pod. | +| `podAnnotations` | `{}` | Annotations on the manager pod. | + +`commonLabels` and `podLabels` must not set the chart's reserved label keys +(`app.kubernetes.io/name`, `app.kubernetes.io/instance`, +`app.kubernetes.io/managed-by`, `app.kubernetes.io/version`, `helm.sh/chart`, +`control-plane`); the chart fails rendering if they do. + +### Kyverno image verification + +This optional block renders a [Kyverno](https://kyverno.io) image-verification +policy that requires the operator images to carry a verifiable Sigstore +signature and SLSA provenance attestation. It is disabled by default and +requires Kyverno to be installed in the cluster. + +| Key | Default | Meaning | +| --- | --- | --- | +| `kyverno.imageVerification.enabled` | `false` | Render the image-verification policy. | +| `kyverno.imageVerification.name` | `""` | Policy name. Empty derives the name from the release. | +| `kyverno.imageVerification.validationFailureAction` | `Enforce` | Policy action when verification fails (`Enforce` or `Audit`). | +| `kyverno.imageVerification.webhookTimeoutSeconds` | `30` | Admission webhook timeout for the policy. | +| `kyverno.imageVerification.imageReferences` | `[]` | Image reference globs the policy applies to. Empty falls back to the manager repository (with `:*` and `@*`). | +| `kyverno.imageVerification.attestor.issuer` | `https://token.actions.githubusercontent.com` | Keyless OIDC issuer for the signing identity. | +| `kyverno.imageVerification.attestor.subject` | `""` | Exact OIDC subject (certificate identity). | +| `kyverno.imageVerification.attestor.subjectRegExp` | `^https://github\.com/meigma/yacd/\.github/workflows/release\.yml@refs/tags/v[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z][0-9A-Za-z.-]*)?$` | OIDC subject regular expression. | +| `kyverno.imageVerification.attestor.rekor.url` | `https://rekor.sigstore.dev` | Rekor transparency log URL. | +| `kyverno.imageVerification.attestation.type` | `https://slsa.dev/provenance/v1` | Predicate type of the required attestation. | +| `kyverno.imageVerification.attestation.buildType` | `https://actions.github.io/buildtypes/workflow/v1` | Expected build type in the provenance. | + +## Manager flags + +Flags accepted by the manager binary, from `cmd/options.go`. The chart sets the +relevant flags for you; configure these directly only when running the manager +outside the chart. + +| Flag | Default | Meaning | +| --- | --- | --- | +| `--metrics-bind-address` | `0` | Metrics endpoint bind address. `0` disables the metrics server. The chart sets `:` when metrics are enabled. | +| `--health-probe-bind-address` | `:8081` | Liveness/readiness probe bind address. | +| `--leader-elect` | `false` | Enable controller-runtime leader election. | +| `--metrics-secure` | `true` | Serve metrics over HTTPS with Kubernetes authn/authz. Negatable as `--no-metrics-secure`. | +| `--enable-http2` | `false` | Enable HTTP/2 for the metrics and webhook servers. Off by default to avoid the HTTP/2 rapid-reset CVEs. | +| `--log-format` | `json` | Log output format. One of `json`, `text`. | +| `--log-level` | `info` | Minimum log level. One of `debug`, `info`, `warn`, `error`. | +| `--metrics-cert-path` | `""` (unset) | Directory holding the metrics server certificate material. Empty disables the certificate watcher. | +| `--metrics-cert-name` | `tls.crt` | Metrics certificate filename within `--metrics-cert-path`. | +| `--metrics-cert-key` | `tls.key` | Metrics private key filename within `--metrics-cert-path`. | +| `--webhook-cert-path` | `""` (unset) | Directory holding the webhook server certificate material. Empty disables the certificate watcher. | +| `--webhook-cert-name` | `tls.crt` | Webhook certificate filename within `--webhook-cert-path`. | +| `--webhook-cert-key` | `tls.key` | Webhook private key filename within `--webhook-cert-path`. | +| `--default-cardano-testnet-image` | `""` | Override the cardano-testnet image used for the create-env init container and the default cardano-node container. Empty uses the built-in versioned reference. | +| `--default-cardano-tools-image` | `""` | Override the cardano-tools image used for artifact staging containers. Empty uses the built-in versioned reference. | diff --git a/docs/reference/environment.md b/docs/reference/environment.md new file mode 100644 index 0000000..4721531 --- /dev/null +++ b/docs/reference/environment.md @@ -0,0 +1,122 @@ +# Environment file + +The environment file is the developer-facing YACD configuration document you +pass to the CLI with `-f/--file` (see [CLI reference](cli.md)). It is a single +YAML document with a fixed `apiVersion`/`kind` envelope and one field of +substance: `spec.network`, a Cardano network specification. + +The CLI loads this file, validates it, and renders it into a `CardanoNetwork` +object that the operator reconciles. The document does **not** carry a name or +namespace. Identity is supplied on the command line (`yacd up NAME -n +NAMESPACE`), so one file can deploy under many names and namespaces. + +## Document shape + +```yaml +apiVersion: yacd.meigma.io/devconfig/v1alpha1 +kind: Environment +spec: + network: {} # a CardanoNetworkSpec +``` + +| Field | Type | Required | Description | +| --- | --- | --- | --- | +| `apiVersion` | string | yes | Must equal `yacd.meigma.io/devconfig/v1alpha1`. | +| `kind` | string | yes | Must equal `Environment`. | +| `spec.network` | object | yes | A `CardanoNetworkSpec`. Decoded directly into the API type, so the document exposes the same fields the CRD exposes. See the [CardanoNetwork reference](cardanonetwork.md) for every network field, type, enum, and default. | + +The envelope is intentionally thin. `spec.network` is the only field today; +the wrapper exists so future top-level fields can be added without breaking +existing documents. + +For complete, copy-paste files per mode and profile, see the +[recipes](../recipes.md). + +## Validation + +The CLI rejects an invalid file before contacting the cluster. Validation runs +in three layers. + +### Strict decoding (unknown fields rejected) + +The file is parsed with a strict YAML decoder. Any key the schema does not +recognize is an error. A misspelled or misplaced field fails the load rather +than being silently ignored. The decoder names the offending key by its leaf +field name, not its full path. + +``` +parse developer config: error unmarshaling JSON: while decoding JSON: json: unknown field "nde" +``` + +### Envelope and shape checks + +After decoding, the document is checked for envelope integrity and structural +consistency: + +- `apiVersion` must equal `yacd.meigma.io/devconfig/v1alpha1`. +- `kind` must equal `Environment`. +- `spec.network.node.version` must be set. +- `spec.network.node.port` must be greater than 0. +- `spec.network.mode` must be `local` or `public`. +- In `local` mode, `spec.network.local` is required and `spec.network.public` + must be absent. +- In `public` mode, `spec.network.public` is required and `spec.network.local` + must be absent. `spec.network.public.profile` must be `preview`, `preprod`, + or `mainnet`. A `bootstrap` block is allowed only for `mainnet`, where + `bootstrap.mithril` is required. + +Runtime-support checks also run here (supported eras, supported Ogmios/Kupo +images and version pairings, port-conflict detection, mainnet storage minimums, +and which chain APIs are allowed per mode). Those constraints belong to the +network spec; see the [CardanoNetwork reference](cardanonetwork.md). + +### Explicit-field enforcement + +Some network fields have CRD defaults. On a strongly-typed Go value the +decoder cannot tell whether the author wrote `port: 0` or omitted `port` +entirely, and a silently-defaulted value here would produce surprising runtime +behavior (for example, an unset `node.port` rendering a Service with port 0). +To prevent that, the CLI re-reads the raw YAML and requires these paths to be +written explicitly. A missing path fails with: + +``` +spec.network.node.port must be set explicitly in developer config +``` + +**Always required:** + +- `spec.network.mode` +- `spec.network.node.version` +- `spec.network.node.port` + +**Required when `mode: local`:** + +- `spec.network.local.networkMagic` +- `spec.network.local.era` +- `spec.network.local.timing.slotLength` +- `spec.network.local.timing.epochLength` +- `spec.network.local.topology.pools.count` + +**Required when `mode: public`:** + +- `spec.network.public.profile` + +**Required only when the parent block is present:** + +| If you include... | You must also set explicitly | +| --- | --- | +| `spec.network.node.storage` | `spec.network.node.storage.size` | +| `spec.network.local.genesis` | `spec.network.local.genesis.profile` | +| `spec.network.chainAPI.ogmios` | `...ogmios.enabled`, `...ogmios.image`, `...ogmios.port` | +| `spec.network.chainAPI.kupo` | `...kupo.enabled`, `...kupo.image`, `...kupo.port` | + +These rules cover only *presence* in the source. The accepted values, types, +and defaults for each field are documented in the +[CardanoNetwork reference](cardanonetwork.md). Chain-API blocks you omit +entirely keep their network-spec defaults; you opt into explicit-field +enforcement for a block only by including that block. + +An ogmios or kupo block may also carry the optional `service` (ClusterIP or +NodePort) and `externalURL` fields to make the endpoint reachable from outside +the cluster; these are not required-explicit. See +[external access](cardanonetwork.md#external-access). diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md new file mode 100644 index 0000000..cb2c212 --- /dev/null +++ b/docs/troubleshooting.md @@ -0,0 +1,259 @@ +# Troubleshooting + +How to diagnose and fix the most common problems with a `CardanoNetwork` or +`CardanoDBSync`. Start by reading the status, then match your symptom to a +failure mode below. + +## Start here: read the status + +For a network, run [`yacd info`](reference/cli.md) with the network name. It +fetches the `CardanoNetwork` and prints its conditions, network identity, and +endpoints: + +```sh +yacd info my-net +``` + +The output has a `Conditions:` section with one line per condition, formatted as +`Type: Status (Reason) - Message`: + +``` +Conditions: + Ready: False (Progressing) - waiting for the primary node to synchronize + NodeReady: True (NodeRunning) - primary node container is running + NodeSynchronized: False (Syncing) - node is catching up to the inferred tip +``` + +Read it top-down: + +- **`Ready`** is the rollup. `Ready: True` means the network is usable through + its published endpoints. `Ready: False` means it is not, and the per-component + conditions tell you which part is blocking. +- **`Degraded: True`** means the resource failed to reach or maintain its + desired state. Read its `Message` first; it carries the specific error. +- **`Progressing`** means the resource is still being created or updated. A + `False` rollup with `Progressing: True` is normal during initial bring-up. + +The per-component conditions a `CardanoNetwork` publishes are `NodeReady` +(primary node container running), `NodeSynchronized` (caught up to the inferred +network tip), `NodeProgressing` (tip advancing or already synchronized), +`ArtifactsReady` (artifact bundle staged and served over HTTP), `OgmiosReady`, +`KupoReady`, and `DBSyncAttachmentReady` (a primary-sidecar db-sync attachment +is not blocking the primary Pod). See the +[CardanoNetwork reference](reference/cardanonetwork.md) for the full list. + +For machine-readable status, including the `reason` and `message` of every +condition, add `--json`: + +```sh +yacd info my-net --json +``` + +For a `CardanoDBSync`, `yacd info` does not apply. Read its conditions directly: + +```sh +kubectl get cardanodbsync my-dbsync -o jsonpath='{.status.conditions}' | jq +``` + +Its component conditions are `Ready`, `FollowerNodeReady`, `NodeSocketReady`, +`SidecarMaterialReady`, `PostgresReady`, `DBSyncReady`, and `Synced`. See the +[CardanoDBSync reference](reference/cardanodbsync.md). + +## Network never reaches Ready + +**Symptom.** `yacd info` shows `Ready: False` and stays there. + +**Cause.** Look at which child condition is `False`. The common cases: + +- `NodeReady: False` — the primary node container is not running yet. Check the + node Pod with `kubectl describe pod` for image pull or scheduling errors (see + [image pull issues](#image-pull-issues) and [storage rejected](#pvc-too-small-or-storage-rejected)). +- `NodeSynchronized: False` with `NodeProgressing: True` — the node is running + but still catching up to the tip. This is expected on public profiles and + resolves on its own once the node syncs. The `sync` status sub-resource + reports `lagSlots` and `lagSeconds` so you can watch progress. +- `OgmiosReady: False` or `KupoReady: False` — a chain API sidecar is enabled + but not yet connected or synchronized. Ogmios and Kupo are enabled by default. + +**Fix.** Wait out a genuine sync, or fix the blocking child condition using the +matching section below. To watch live: + +```sh +kubectl get cardanonetwork my-net -w +``` + +## Mainnet stuck without a Mithril bootstrap + +!!! warning "Mainnet requires a Mithril bootstrap" + Mainnet is too large to sync from genesis in a development context. The + `CardanoNetwork` CRD validation rejects a mainnet network that has no + Mithril bootstrap, so the API server refuses the resource before it ever + reaches the controller. + +**Symptom.** Applying a `public` network with `spec.public.profile: mainnet` +fails immediately with: + +``` +bootstrap.mithril is required only when public.profile is mainnet +``` + +**Cause.** The CRD enforces that `spec.public.bootstrap.mithril` is present +exactly when (and only when) the profile is mainnet. A mainnet network with no +bootstrap, or a preprod/preview network that sets a bootstrap, is invalid. + +**Fix.** Add a Mithril bootstrap block. The image and snapshot have defaults, so +an empty block is enough to satisfy the rule: + +```yaml +spec: + mode: public + public: + profile: mainnet + bootstrap: + mithril: {} +``` + +`mithril.snapshot` defaults to `latest` and `mithril.image` defaults to the +pinned `mithril-client` image. See the +[CardanoNetwork reference](reference/cardanonetwork.md) for the bootstrap +fields. Mainnet also needs large persistent storage and the host-only opt-in to +create it; see the [recipes](recipes.md) for a complete mainnet manifest. + +## PVC too small or storage rejected + +**Symptom.** `NodeReady: False`, and the node Pod is stuck `Pending`. A +`kubectl describe pod` shows the PVC is not bound, or events report a +provisioning failure. + +**Cause.** The node database PVC requested by `spec.node.storage.size` could not +be provisioned. Typical reasons: the requested size exceeds what the cluster can +provision, the named `spec.node.storage.storageClassName` does not exist, or the +default StorageClass cannot bind the claim. The node storage size defaults to +`10Gi`, which is fine for local devnets but far too small for a public profile. + +**Fix.** Inspect the claim and its events: + +```sh +kubectl get pvc -l app.kubernetes.io/instance=my-net +kubectl describe pvc +``` + +Then set a size and StorageClass the cluster can satisfy: + +```yaml +spec: + node: + storage: + size: 200Gi + storageClassName: standard +``` + +A PVC's requested size is immutable after creation. If you need to grow it, +either use a StorageClass with `allowVolumeExpansion: true` or recreate the +network. See [`spec.node.storage`](reference/cardanonetwork.md) for the fields. + +## External Postgres unreachable (db-sync) + +**Symptom.** `CardanoDBSync` shows `PostgresReady: False` or `DBSyncReady: +False`, and `Ready` never becomes `True`. The db-sync container logs report +connection refused, authentication failure, or TLS errors against the Postgres +host. + +**Cause.** With an external database, db-sync connects to the host you supplied +in `spec.database.external`. The connection fails when the host or port is +wrong, the password Secret is missing or holds the wrong key, the database or +user does not exist, or `sslMode` does not match the server's TLS posture. + +**Fix.** Verify each field against the running Postgres: + +- `spec.database.external.host` and `port` (default `5432`) resolve from inside + the cluster. +- `spec.database.external.database` (default `cexplorer`) and `user` (default + `postgres`) exist on the server. +- `spec.database.external.passwordSecretRef` names a same-namespace Secret, and + the referenced `key` (default `password`) holds the password. +- `spec.database.external.sslMode` (default `disable`) matches the server. Use + `require`, `verify-ca`, or `verify-full` only if the server presents TLS. + +Confirm the Secret exists and has the expected key: + +```sh +kubectl get secret -o jsonpath='{.data}' | jq 'keys' +``` + +Note that `spec.database` must set exactly one of `external` or `managed`; +setting both, or neither, is rejected at admission with +`exactly one of database.external or database.managed must be set`. See the +[CardanoDBSync reference](reference/cardanodbsync.md) for the database fields. + +## Wallet funding fails + +**Symptom.** `yacd wallet add --topup` or `yacd wallet topup` fails to build, +submit, or confirm a funding transaction. + +**Cause and fix.** + +- **No `faucet` wallet to fund from.** Only `local` networks get a + genesis-funded `faucet` wallet. On a public network there is nothing to spend + from by default — fund the target with `--from` a wallet you have funded + yourself. +- **The source wallet has insufficient funds.** Funding spends real UTxOs. Check + the source with `yacd wallet list `, then top it up (or use `--from` a + funded wallet) before funding others. +- **`--await` times out.** Confirmation polls Kupo. If the network is still + syncing or `KupoReady` is `False`, the output may not appear in time; raise + `--await-timeout` or retry once the network is Ready. + +See the [funding guide](developer/funding.md) and the +[`yacd wallet` reference](reference/cli.md#wallet). + +## devnet host port already in use + +**Symptom.** `yacd devnet` fails to create its cluster with a host-port collision +error naming port `1337` or `1442`. + +**Cause.** `yacd devnet` maps Ogmios to host port `1337` and Kupo to `1442` so +they are reachable on `localhost` without a port-forward. If another process +(another devnet, a local Ogmios/Kupo, or anything else) already holds one of +those ports, k3d cannot bind it. + +**Fix.** Free the port — stop the other process, or `yacd devnet down` a previous +devnet — then re-run `yacd devnet`. + +!!! note "An advertised `externalURL` that is not reachable" + `run` and the wallet commands probe a network's `externalURL` before using + it and fall back to a port-forward when the probe fails, so an unreachable + `externalURL` only adds a brief delay, never an error. To force a specific + endpoint, pass `--ogmios-url` / `--kupo-url` (funding) or set + `YACD_OGMIOS_URL` / `YACD_KUPO_URL`. + +## Image pull issues + +**Symptom.** A Pod is stuck `ImagePullBackOff` or `ErrImagePull`, and the owning +condition (`NodeReady`, `OgmiosReady`, `KupoReady`, or +`PostgresReady`/`DBSyncReady`) stays `False`. + +**Cause.** Kubernetes could not pull a container image. The reference is wrong, +the tag or digest does not exist, or the registry needs credentials the cluster +does not have. + +**Fix.** Identify the failing image from the Pod events: + +```sh +kubectl describe pod +``` + +Then check the override on the spec. Each component has a pinned default image, +so a pull failure usually means a custom override is wrong: + +- `spec.node.image` / `spec.node.version` (the controller derives the node image + from `version`, default `11.0.1`, when `image` is unset). +- `spec.chainAPI.ogmios.image` and `spec.chainAPI.kupo.image` (pinned defaults). +- `spec.public.bootstrap.mithril.image` for the Mithril client. +- For db-sync: `spec.image` and `spec.database.managed.image` (default + `postgres:17.2-alpine`). + +If the image is private, add an `imagePullSecret` to the namespace. See the +[CardanoNetwork reference](reference/cardanonetwork.md) and +[CardanoDBSync reference](reference/cardanodbsync.md) for the exact field paths +and pinned default tags. diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 0000000..e442252 --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,76 @@ +site_name: yacd +site_description: Kubernetes-native Cardano development environments +repo_url: https://github.com/meigma/yacd +repo_name: meigma/yacd +docs_dir: docs + +theme: + name: material + palette: + # Automatic mode follows the operating system preference (the default). + - media: "(prefers-color-scheme)" + toggle: + icon: material/brightness-auto + name: Switch to light mode + # Light mode. + - media: "(prefers-color-scheme: light)" + scheme: default + toggle: + icon: material/brightness-7 + name: Switch to dark mode + # Dark mode. + - media: "(prefers-color-scheme: dark)" + scheme: slate + toggle: + icon: material/brightness-4 + name: Switch to system preference + features: + - navigation.top + - navigation.footer + - navigation.tracking + - toc.follow + - content.code.copy + - content.code.annotate + - search.suggest + - search.highlight + +markdown_extensions: + - admonition + - attr_list + - md_in_html + - tables + - toc: + permalink: true + - pymdownx.highlight: + anchor_linenums: true + - pymdownx.inlinehilite + - pymdownx.snippets + - pymdownx.superfences + - pymdownx.tabbed: + alternate_style: true + +plugins: + - search + +nav: + - Home: index.md + - Developer Guide: + - Getting started: developer/getting-started.md + - Defining networks: developer/networks.md + - Funding accounts: developer/funding.md + - Connecting tools & tests: developer/connecting-tools.md + - Operator Guide: + - Installation: operator/installation.md + - DB-Sync indexing: operator/db-sync.md + - Testing & CI: operator/testing.md + - Reference: + - CLI: reference/cli.md + - CardanoNetwork: reference/cardanonetwork.md + - CardanoDBSync: reference/cardanodbsync.md + - Environment file: reference/environment.md + - Configuration: reference/configuration.md + - Recipes: recipes.md + - Troubleshooting: troubleshooting.md + - Concepts: + - Architecture: concepts/architecture.md + - Security model: concepts/security.md diff --git a/moon.yml b/moon.yml index bd46d0d..175cafe 100644 --- a/moon.yml +++ b/moon.yml @@ -165,3 +165,17 @@ tasks: options: cache: false runInCI: true + + docs: + command: 'uv run --with mkdocs-material==9.7.6 mkdocs build --strict' + inputs: + - 'docs/**/*' + - 'mkdocs.yml' + options: + cache: false + + docs-serve: + command: 'uv run --with mkdocs-material==9.7.6 mkdocs serve' + options: + cache: false + runInCI: false