fix: make Zallet config readable by the container uid (1000)#6
Closed
alchemydc wants to merge 64 commits into
Closed
fix: make Zallet config readable by the container uid (1000)#6alchemydc wants to merge 64 commits into
alchemydc wants to merge 64 commits into
Conversation
* Add project brief with core features and target users * Define technical preferences and stack components * Document main services: Zebra, Zaino, and Zallet
* Define core problems with existing Zcashd implementation * Document proposed modular solution architecture * Specify target user groups and their requirements * Detail key user experience goals including observability * Outline security and reliability requirements Relates to #1
* Add detailed Product Context - Document problems with current Zcashd implementation - Define modular architecture with Zebra, Zaino, and Zallet - Specify user experience goals and target users - Add initial key metrics for adoption * Expand Technical Context - Document dependencies for all components - Detail common dependencies across stack - Add comprehensive tool usage patterns - Document build and testing workflows This improves project documentation by providing clear context for both product and technical aspects of the Z3 stack. Relates to #1
* Update zaino submodule to latest main - Sync with upstream zingolabs/zaino - Pull latest gRPC protocol changes - Update librustzcash dependency * Update zebra submodule to latest main - Sync with upstream ZcashFoundation/zebra - Include latest consensus fixes - Update network protocol handling This keeps our dependencies in sync with upstream development.
* Add cross-references between key documents - Link Project Brief to Product Context - Link Product Context to Technical Context - Link Technical Context to System Patterns - Link System Patterns to Progress document * Update README navigation - Add links to all key documents - Include brief description for each - Maintain consistent formatting This improves documentation navigation and helps users find related information across the Z3 documentation. Relates to #1
* Add zcashd codebase for reference purposes - Add submodule at path zcashd/ - Configure .gitmodules with upstream repo - Use latest stable release tag This adds the legacy codebase we're replacing as a submodule to help ensure compatibility and feature parity during development. Relates to #1
* Add links to RPC method documentation - Add link to April 2025 RPC method usage report - Update reference to Google Sheet source - Improve readability of RPC section * Organize documentation structure - Move RPC data to separate files - Use consistent path references - Maintain documentation hierarchy This improves navigation and accessibility of RPC method documentation for the Z3 stack components. Relates to #1
* Update zebra submodule - Bump to latest main branch - Include recent consensus fixes - Update RPC method implementations * Update zaino submodule - Bump to latest main branch - Improve indexing performance - Update lightwalletd compatibility This keeps Z3 in sync with latest upstream changes from both components. Relates to #1
* Reorganize method categories - Group by functional area - Add descriptions for each method - Maintain consistent formatting - Follow markdown style guide * Add new categories - Debug and Testing - Address and Validation - Network and Node Info This improves readability and organization of the RPC method reference documentation. Relates to #1
Add z3_docker.md summarizing Zebra and Zaino Docker configurations and unified compose plan Update activeContext.md with Docker orchestration focus, decisions, and patterns Update progress.md with detailed status of Docker integration work and next steps Clarify next steps for Zallet integration and production readiness
Also add a utility script to pull Zebra node count from zcashblockexplorer.com. Scrapes HTML because no API afaict.
…ay documentation * Update techContext.md to improve formatting and clarity of technology descriptions. * Add new `z3_component_interplay.md` to detail the roles, interactions, and dependencies of Zebra, Zaino, and Zallet within the Z3 stack.
…images * Introduced a new workflow to build Z3 images * Created a sub-workflow for building Docker images with customizable inputs, including repository, Dockerfile path, and Rust parameters.
* Renamed the existing build job to 'build-zaino' for clarity. * Introduced a new job 'build-zallet' to build the zallet Docker image with specified permissions and parameters.
…delines Introduces a structured approach to Z3 service orchestration using Docker Compose. - Adds detailed README.md instructions for setting up and running the Z3 stack via Docker Compose, including .env configuration, certificate/identity file generation, and operational commands. - Implements .gitignore rules to correctly manage the 'config' directory, ensuring essential subdirectories (like 'config/tls') can be tracked via .gitkeep files while ignoring other contents, which is crucial for Docker volume mounts and config sources. - Includes a minor cosmetic update to the name of the GitHub Actions workflow for building Z3 images. This commit provides a foundational, well-documented Docker setup for deploying and managing the Zebra, Zaino, and Zallet services cohesively.
…cks (#4) * refactor: modernize Z3 stack with upstream repos and Zebra health checks Update the Z3 stack to use official upstream repositories and their corresponding Dockerfiles instead of building from personal forks with specific branches. This refactor also implements proper Zebra configuration using its new environment variable system and health check endpoints. Major changes: 1. Use official upstream repositories with default branches - Zebra: ZcashFoundation/zebra (main branch) - Zaino: zingolabs/zaino (dev branch) - Zallet: zcash/wallet (main branch) - Updated Dockerfile paths to match each project's structure 2. Implement Zebra's environment variable configuration system - Use config-rs format: ZEBRA_SECTION__KEY for Zebra configuration - Implement Z3_* prefix for Docker Compose infrastructure variables - Three-tier variable hierarchy to prevent config collisions: * Z3_* for infrastructure (volumes, ports, Docker Compose only) * Shared variables (NETWORK_NAME, etc.) remapped per service * Service-specific (ZEBRA_*, ZAINO_*, ZALLET_*) for app config 3. Leverage Zebra's new health check endpoints - Use /ready endpoint for production (synced near network tip) - Use /healthy endpoint for development (has peer connections) - Implement two-phase deployment pattern for blockchain sync - Configure healthcheck with 90s start_period for cached state 4. Improve security and documentation - Add fix-permissions.sh for proper UID/GID ownership setup - Document specific UIDs/GIDs for each service - Add warnings about unstable development branches - Update deployment documentation for fresh sync scenarios 5. Remove hardcoded paths and configurations - Eliminate user-specific paths from docker-compose.yml - Use environment variables for all customizable settings - Add docker-compose.override.yml.example for development mode * fix(compose): workaround through zallet limitations * fix(zallet): Use Zebra as the indexer, and update docs * chore: reduce logging * docs: update README.md --------- Co-authored-by: Gustavo Valverde <gustavovalverde@unknowncad5ab0a9f79.lan>
* feat(docker): configuration and documentation for ARM64 support - Updated `.env` to include a configuration option for ARM64 users, allowing native builds by uncommenting the `DOCKER_PLATFORM` variable. - Modified `docker-compose.yml` to utilize the `DOCKER_PLATFORM` variable for platform-specific builds. - Expanded `README.md` with a detailed section on platform configuration, including instructions for enabling native ARM64 support and system requirements. - Clarified Zallet's connection to Zebra's JSON-RPC endpoint in `zallet.toml` and updated related documentation. * chore: update submodule references
… and `OWNER_GID` instead (#5)
Co-authored-by: David Campbell <alchemydc@users.noreply.github.com>
1) generate zallet_identity when running init.sh for regtest testing, rather than checking into git. 2) clarify tagging of local vs remote images in the main z3 docker-compose.yml 3) bump Zebra to 4.1.0 in the regtest docker-compose.yml. 4) modify init to dynamically generate the age keys required by zallet for regtest mode, and also dynamically generate the pwhash for the rpc user. this is done mostly to reduce noise from automated code scanning tools that search for leaked keys. 5) update README for the rpc-router 6) remove testing zallet-identity.txt age key from git 7) remove default pw salt and hash from zallet.toml
Add the OpenRPC router
Bumps [rustls-webpki](https://github.com/rustls/webpki) from 0.103.9 to 0.103.10. - [Release notes](https://github.com/rustls/webpki/releases) - [Commits](rustls/webpki@v/0.103.9...v/0.103.10) --- updated-dependencies: - dependency-name: rustls-webpki dependency-version: 0.103.10 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>
…er/rustls-webpki-0.103.10 chore(deps): bump rustls-webpki from 0.103.9 to 0.103.10 in /rpc-router
… stack (#22)
* feat(regtest): add zaino service to regtest stack
Zaino serves as the lightwalletd-compatible gRPC interface for light
wallet clients (e.g. Zingo). It connects to Zebra's RPC and exposes:
- gRPC (port 8137): Compact Block Protocol, direct access for clients
- JSON-RPC (port 8237): overlapping with Zebra, available for tooling
gRPC is exposed directly and not routed through the rpc-router since
it is a different protocol (HTTP/2 + protobuf) from the JSON-RPC router.
* refactor(docker): redesign compose architecture with defaults-in-compose pattern
Replaces the previous .env-dependent setup with a self-sufficient docker-compose.yml
where every variable has a ${VAR:-default} fallback. The stack now works on a fresh
clone with zero configuration files.
Architecture changes:
- All variables in docker-compose.yml use ${VAR:-default} (Mainnet defaults)
- .env is gitignored and optional (users create it only to override)
- Single .env.example replaces three .env.example.{mainnet,testnet,regtest} files
- Regtest uses a compose overlay (docker-compose.regtest.yml) merged via
COMPOSE_FILE in .env.regtest, with COMPOSE_PROJECT_NAME for volume isolation
- Removed standalone regtest/docker-compose.yml (127 lines of duplicated
service definitions); the overlay is ~60 lines of structural differences only
- Dissolved regtest/ directory: configs moved to config/regtest/, init script
to scripts/regtest-init.sh, docs to docs/regtest.md
- Flattened config/zaino/zindexer.toml to config/zaino.toml (1-file directory)
- Moved check-zebra-readiness.sh and fix-permissions.sh to scripts/
Docker Compose improvements:
- x-common YAML anchor for shared log rotation + security hardening
- cap_drop: [ALL] on all services (Zebra adds back CHOWN/SETUID/SETGID/etc.
for its setpriv-based entrypoint)
- security_opt: [no-new-privileges:true] on all services
- Log rotation (50MB x 5 files) prevents unbounded disk growth
- stop_grace_period on all services (30s Zebra, 15s others)
- start_interval: 5s for faster startup healthcheck detection
- Zaino healthcheck upgraded from zainod --version (useless process check)
to bash /dev/tcp port check
- Image override variables: ZEBRA_IMAGE, ZAINO_IMAGE, ZALLET_IMAGE
- Removed env_file from Zallet (doesn't support env var config) and limited
Zebra's env_file to optional config-rs passthrough (metrics, OpenTelemetry)
- Regtest overlay uses environment: !override on Zaino to switch from cookie
auth to username/password auth (Zaino blocks passwords in env vars)
- Regtest rpc-router RPC_PASSWORD parameterized via ${RPC_PASSWORD:-zebra}
Submodule bumps:
- Zebra: bumped to latest main (setpriv replaces gosu, enables cap_drop)
- Zaino: bumped to latest dev
- Zallet: bumped to latest main
Documentation:
- README restructured with progressive disclosure (<details> sections)
- Mermaid architecture diagram showing Zallet's embedded Zaino libs
- docs/docker-architecture.md: rationale for all compose patterns
- docs/regtest.md: regtest workflow, test commands, monitoring setup
- All pre-built images used by default; no build step in Quick Start
Bugs fixed during validation:
- config-rs crashes on empty string env vars (metrics, OpenTelemetry)
- macOS openssl dgst output format (awk $2 vs $NF)
- Zallet init-wallet-encryption missing --config flag in compose run
- Zaino config-rs duplicate field panic (TOML + env var conflict)
- ZEBRA_HEALTH__ENFORCE_ON_TEST_NETWORKS was hardcoded, now overridable
* fix(docker): correct zaino volume mount, command, and capabilities
The zaino service had three issues:
- Volume mounted to /home/zaino/.cache/zaino, a path that doesn't
exist in the container (user is container_user, not zaino). Data
was written to /app/data via symlink but never persisted.
Mount to /app/data to match the Dockerfile's published interface.
- Command included "zainod" which duplicated the binary invocation
since the entrypoint already calls zainod. Remove it so CMD only
passes arguments.
- Missing cap_add for the entrypoint's privilege-dropping workflow.
The x-common anchor drops all capabilities, but zaino's entrypoint
needs CHOWN/DAC_OVERRIDE/FOWNER for directory setup and
SETUID/SETGID for setpriv, matching zebra's configuration.
Fixes #15
* fix(regtest): replace committed pwhash with placeholder
The regtest zallet config had a real generated hash committed.
Replace with __GENERATED_BY_INIT_SH__ placeholder that
regtest-init.sh detects and replaces on first run.
The init script now skips hash generation if the placeholder
is already replaced, matching the idempotent pattern used
for identity generation.
* Regtest fixes after PR24 (#25)
* fix(regtest): add Nu5 nuparam and auto-generate TLS certs in init script
- Add Nu5 (c2d6d0b4) to regtest_nuparams in config/regtest/zallet.toml.
Without it, activation_height(Nu5) returns None and the orchard_shardtree
migration panics with an unwrap on None.
- Remove stale wallet.db in regtest-init.sh before init-wallet-encryption
so a previous interrupted run (e.g. from the cookie-mount bug) doesn't
leave a schema-mismatched database that causes 'no such table' errors.
- Tighten ALREADY_INIT check to look for .age files only (written by
generate-mnemonic on successful completion) rather than wallet.db.
- Auto-generate the Zaino TLS certificate in regtest-init.sh when missing,
so 'docker compose up' no longer fails with a bind-mount path error.
---------
Co-authored-by: Gustavo Valverde <g.valverde02@gmail.com>
- Bump Zebra to v4.3.0 - Bump Zaino to zingolabs/zaino@dev (0.2.0, Zebra 4.3.0 compatible) - Bump Zallet to zcash/wallet@main - Fix zaino command: add missing `start` subcommand (fixes #28) - Rebuild zaino CI image (sha-83e41d7) Closes #28
…#30)
* ci: add PR validation, smoke tests, and workflow security hardening
- Add CI workflow: compose config validation + service startup smoke tests on every PR
- Add integration test workflow: regtest stack end-to-end validation on dev push
- Add zizmor workflow: GitHub Actions security scanning (SARIF → Security tab)
- Add Dependabot for GitHub Actions (monthly, with cooldown)
- Pin all actions to commit SHAs and update to latest versions
- Harden all workflows: persist-credentials: false, permissions: {}t
The smoke tests would have directly caught #28 (missing zaino start subcommand)
Users following the README modify config/zallet.toml (network setting) and regtest-init.sh modifies config/regtest/zallet.toml (pwhash). These tracked files caused conflicts on git pull. - Ship .default templates (tracked) alongside gitignored live copies - regtest-init.sh and CI seed from .default automatically - zaino configs stay tracked (never modified by users/scripts)
feat: add optional zcashd compose profile, bump zebra version, add SAN to Zaino cert.
The repo default branch was renamed dev -> main (issue #26). Update the three workflows that pin push triggers to the old branch name so they fire on the new default. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
zingolabs/zaino's Dockerfile (on dev) declares RUST_VERSION as a build-arg with no default by design — the canonical source is rust-toolchain.toml. Our reusable build workflow only passed SHORT_SHA, so the FROM stage expanded to "rust:-bookworm" and Docker rejected it with "invalid reference format". Add an opt-in `read_rust_version` input to sub-build-docker-image.yaml that reads the channel from rust-toolchain.toml in the just-checked-out upstream repo, validates it as x.y or x.y.z (matching upstream's get-rust-version.sh), and appends it to build-args. The zaino caller opts in; zebra and zallet are unaffected. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Zizmor flagged the previous commit's `Read RUST_VERSION` step for
expanding `${{ inputs.repository }}` and `${{ inputs.ref }}` directly
into the shell `run:` body. Move both into `env:` so they're consumed
as ordinary shell variables, removing the template-expansion sink.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ci: switch push triggers from dev to main
…figs fix: gitignore mutable zallet configs to prevent git pull conflicts
rpc-router has 13 unit tests and 7 integration tests that were not exercised by CI — the existing smoke-test job only builds the binary and validates the docker compose stack. This blind spot meant Cargo.lock bumps (including transitive openssl / rustls-webpki updates) had no automated runtime signal. Add a sibling job that runs `cargo test --locked --all-targets` against rpc-router on Rust 1.85, matching the toolchain pinned in rpc-router/Dockerfile. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ci: add cargo test job for rpc-router
…ter (#34) Bumps [rustls-webpki](https://github.com/rustls/webpki) from 0.103.10 to 0.103.13. - [Release notes](https://github.com/rustls/webpki/releases) - [Commits](rustls/webpki@v/0.103.10...v/0.103.13) --- updated-dependencies: - dependency-name: rustls-webpki dependency-version: 0.103.13 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [openssl](https://github.com/rust-openssl/rust-openssl) from 0.10.75 to 0.10.80. - [Release notes](https://github.com/rust-openssl/rust-openssl/releases) - [Commits](rust-openssl/rust-openssl@openssl-v0.10.75...openssl-v0.10.80) --- updated-dependencies: - dependency-name: openssl dependency-version: 0.10.80 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(docker): bump default zebra image to latest Zebra v5.0.0 was released and tagged `latest`. Switch the docker-compose default and its mirrored references in .env.example, README, and docker-architecture.md from `zfnd/zebra:4.3.1` to `zfnd/zebra:latest` so fresh stacks track upstream Zebra releases without an .env override. Operators who need a pinned version can still set ZEBRA_IMAGE in .env (see the README override example). Note: zaino is pinned to a Zebra-4.3-compatible image (see comment at docker-compose.yml:69); the new zebra default may surface RPC-compat issues that are out of scope for this commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(submodule): bump zebra to v5.0.0 Move the zebra/ submodule pointer from v4.3.0 to the v5.0.0 release tag so the build-from-source path (`docker compose build`) produces a v5.0.0 image consistent with the new default in the previous commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…n guides (#43)
* feat: introduce platform contract with versioned identifiers and CI validators
Adds z3-contract.yaml as the machine-readable inventory of every stable
identifier downstream consumers depend on: networks, Compose project names,
external network names, volume names, in-network DNS, the per-network port
matrix with profile gating, healthcheck transport and endpoint per service,
the env-var schema with namespace tags, and the RPC auth mode per network.
z3-contract.schema.json constrains the YAML's shape so consumers in any
language can validate before integrating. Three CI gates enforce the surface
on every PR:
- validate-contract.py: rendered Compose output matches the contracted
port matrix, volumes, healthcheck transport, and scrape target.
- validate-contract-parity.py: env-var parity across compose files,
.env.example, and the contract YAML (with ecosystem_vars unioned).
- jsonschema: YAML validates against the JSON Schema.
The contract carries its own SemVer version (contract_version), separate
from the repo's release version. Bumping it is a breaking change to
downstream integrators, not to operators. docs/contract.md walks the file-
ownership model and the full surface.
* feat: reshape Compose stack into per-network projects with auto-loaded overrides
Mainnet, testnet, and regtest now run as three Compose projects (z3-mainnet,
z3-testnet, z3-regtest) produced from one base compose plus a per-network
overlay. Host ports for services without an upstream per-network convention
use a +10000 offset relative to mainnet. The three networks coexist on one
host without binding collisions, including under the monitoring and zcashd
profiles.
Per-host operator overrides live in docker-compose.<network>.override.yml,
gitignored. scripts/setup-network.sh <network> copies an empty placeholder
from the tracked .example if the live file is missing, so testnet and
regtest auto-load the override without erroring on a fresh clone. The same
pattern covers per-network Zallet, Zaino, and the regtest Zebra TOML
configs and the Zallet identity: each tracked as .example, copied on first
run, edited freely without git-pull conflicts.
Regtest's Zebra config (config/regtest/zebra.toml.example) activates NU5/NU6
at the heights Zaino's regtest defaults expect, so the stack boots end-to-end
on a fresh clone instead of stalling at Canopy.
Image defaults bumped: Zebra to 5.0.0 (multi-arch; runs native arm64 on
Apple Silicon), Zaino moved to zingodevops/zainod with a semver tag instead
of a commit SHA, zcashd to v6.20.0. All pins remain ${VAR:-tag} fallbacks
that operators override per-pin via Z3_*_IMAGE.
scripts/fix-permissions.sh corrects the bind-mount uid:gid map for Zallet
and renames the printed env var for the Zebra service to Z3_CHAIN_DATA_PATH
to match the compose. scripts/regtest-init.sh mines two blocks (NU5
activation height). scripts/check-zebra-readiness.sh prints the correct
--env-file in its success message per health port.
* docs: add integration archetypes and rewrite operator docs around the contract
Downstream services have three documented attachment archetypes under
docs/integrations/:
- compose-peer: a Docker service in the same logical stack attaches to
the external network and cookie volume by name.
- host-side-pointer: a host process connects via published host ports
and reads the cookie out of the named volume.
- lightwalletd-client: a wallet or scanner dials Zaino's gRPC port
over TLS.
Each guide carries mainnet/testnet/regtest port tables, working code
snippets in Rust and TypeScript, and the regtest auth divergence called
out (rpc_auth.mode: username_password, rpc-router default credentials,
direct-Zebra credentials path) so a consumer can target either auth mode.
README, FAQ, regtest, and docker-architecture docs rewritten to align with
the per-network project model, the override convention, and the multi-arch
Zebra default. Observability READMEs reflect the monitoring profile port
matrix per network.
* fix: unblock CI and apply PR #43 review feedback
The "Contract validation (all networks)" job and a fresh `git clone && docker
compose up` both failed because .env.testnet and .env.regtest listed a
gitignored override file in COMPOSE_FILE. Drop it from COMPOSE_FILE; per-host
overrides are now opt-in via -f or an operator-local append.
Remove the baggage reviewers flagged:
- Zaino runs the upstream -no-tls image; the self-signed cert, the configs
block, and config/tls are gone. Intra-container gRPC is plaintext, and edge
TLS belongs at a reverse proxy.
- Compose no longer pins the json-file logging driver, so the operator's daemon
default (journald and others) wins; daemon.json rotation is documented.
- The dead Zebra indexer port (8230) is removed; nothing in the stack used it.
- Regtest zcashd nuparams are hardcoded and the six activation-height vars
dropped.
Publish Zebra p2p on mainnet and testnet for inbound peers (regtest stays
peerless). Keep image pins for consensus safety and add a Renovate config that
raises bump PRs. regtest-init now requires Compose v2 with a clear message, and
the contract validator gained a bidirectional no-drift check.
Restructure the docs around two audiences with progressive disclosure:
operators (the three commands, production readiness, data and backup) and
developers and testers (multi-network behavior, integration, the contract). The
platform contract is no longer in the operator path, which is what confused the
reviewer. Delete the stale memory-bank and data scratch directories.
Cookie auth removal and rpc-router promotion are deferred to #44.
* fix(ci): stage regtest zebra.toml for the regtest smoke job
The regtest overlay bind-mounts config/regtest/zebra.toml into Zebra, but the
smoke job's staging step only copied the zallet and zaino templates. With the
source file missing, Docker mounted an empty directory and Zebra exited with
"configuration file not found". The job had never run before (it was skipped
behind the failing contract-validation gate), so the gap was latent until that
gate started passing.
* fix: correct zcashd comparator user and datadir for the zodlinc image
The zcashd service still carried `user: "999:999"` and a `/home/zcash/.zcash`
data mount from a previous image. The current zodlinc/zcashd image runs as the
zcash user (uid 2001) with its datadir under /srv/zcashd, so the entrypoint
(forced to uid 999) hit "touch: cannot touch '.zcash/zcash.conf': Permission
denied" and crash-looped. Run as 2001 and mount the data volume at /srv/zcashd,
which the image owns as zcash, so a fresh named volume inherits writable
ownership. fix-permissions.sh now chowns to 2001 to match. The regtest smoke
job surfaced this once the contract-validation gate stopped skipping it.
* refactor: remove the zcashd comparator
The optional zcashd profile is gone: the service, the zcashd Compose profile,
the ZCASHD_* / Z3_ZCASHD_* env vars, the data volume, the contract entries (DNS,
volume, port, healthcheck, profile, image platform), the regtest smoke step, and
the unused zcashd submodule. docker-compose.testnet.yml held only the zcashd
testnet command, so it is deleted too; .env.testnet now loads just the base
compose and selects the network with Z3_NETWORK.
zcashd is deprecating (no releases past v6.20.0) and both PR reviewers questioned
keeping it. It was also broken with the current zodlinc/zcashd image: that image's
entrypoint builds zcashd's invocation from ZCASHD_* env vars and appends the
compose command after it, so the -regtest and -nuparams flags never applied and
the comparator crash-looped. Reworking the stack around that interface for a
deprecating component is not worth it.
The stack is now Zebra + Zaino + Zallet, plus the regtest-only rpc-router and the
monitoring profile.
* fix(regtest): align NU6 and NU6.1 activation heights across Zebra, Zaino, Zallet
Addresses the PR #43 review comments on the regtest network-upgrade list. The
three components disagreed: Zaino's hardcoded regtest anchor
(ZEBRAD_DEFAULT_ACTIVATION_HEIGHTS) activates NU6 at height 2 and NU6.1 at 1000,
but Zebra declared only NU6=2 and Zallet's regtest_nuparams stopped at NU5. A
regtest chain past height 1000 would desync the wallet's view of the chain.
Add NU6 (c8e71055) to Zallet at height 2, and NU6.1 (4dec4df0) to both Zebra and
Zallet at height 1000, matching Zaino. NU6.2 stays out on purpose: neither Zaino
0.4.0-rc.2 nor Zallet's zcash_protocol 0.7.2 defines it, so a 5437f330 entry
would fail to parse; it waits until all three pinned versions know it.
Verified at runtime: regtest boots with the new config (Zebra parses the quoted
"NU6.1" = 1000 key) and Zaino reads the chain Zebra produces.
A plain `git clone` now boots the full stack: the default compose pulls pinned images and needs no source checkout. The zebra, zaino, and zallet submodules are removed, and the from-source build path moves to an opt-in overlay (docker-compose.build.yml) whose contexts point at a gitignored vendor/ dir. scripts/vendor.sh clones each upstream repo there at the tag matching its image pin. Also deletes the build-z3-images workflow and its sub-build helper. It only triggered when its own file changed and pushed registry images that nothing in compose consumed. Docs drop every git submodule instruction in favor of vendor.sh plus the build overlay, and the gRPC proto examples point at vendor/zaino/zaino-proto. Adds a TCP-tuning production note referencing #36.
Zallet runs on a distroless image pinned to user "1000:1000" (no shell,
entrypoint, or capabilities to chown/override at runtime). It reads two
bind-mounted host files, zallet.toml and zallet_identity.txt, whose host
ownership/mode pass straight through. Operators whose host uid \!= 1000 hit:
zallet fatal error: config error: path error: /etc/zallet/zallet.toml:
Permission denied (os error 13)
because both files end up mode 0600 owned by the host uid: setup-network.sh
chmod 600s the age key, and regtest-init.sh rewrites zallet.toml via
mktemp+mv, leaking mktemp's 0600 onto the config.
Decouple host uid from container uid the same way the cookie-permissions
sidecar already does for Zebra's RPC cookie, rather than parametrizing the
pinned uid (which would force operators to also solve data-volume ownership
that distroless Zallet cannot self-heal):
- setup-network.sh: chmod 644 the copied non-secret TOMLs; grant the age key
read to uid 1000 only via `setfacl -m u:1000:r` (stays 0600 otherwise),
with a warning + chmod 644 fallback when setfacl/acl is unavailable.
- regtest-init.sh: chmod 644 zallet.toml after the pwhash rewrite.
- docs + README: document that uid \!= 1000 operators need no uid coordination
for Zallet config; note the acl/setfacl soft prerequisite.
- ci.yaml: reproduce the scripts' permission end-state in regtest staging and
assert a --user 1000:1000 container can read both config files.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Owner
Author
|
Superseded by ZcashFoundation#46 (opened against the upstream repo so CI runs on the PR). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On a bare-metal regtest bring-up where the host user's uid ≠ 1000, Zallet dies at startup:
Zallet bind-mounts two operator-authored host files read-only (
config/<network>/zallet.tomlandconfig/<network>/zallet_identity.txt). Bind mounts pass host ownership/mode straight through, both files end up mode 0600 owned by the host uid, and the Zallet container runs as uid 1000 — so it can't read them.Why the files are 0600:
zallet_identity.txt—setup-network.shrunsrage-keygenthenchmod 600.zallet.toml(regtest) —regtest-init.sh'supdate_zallet_rpc_pwhash()rewrites the file viamktemp+mv;mktempcreates the temp file mode 0600 andmvcarries that onto the config.Why only Zallet hits this: it's the only core service on a distroless image (no shell, entrypoint, or capabilities), so it's pinned with
user: "1000:1000"and runs as that uid from PID 1. Zebra (root entrypoint →setpriv) and Zaino (root +cap_add: [DAC_OVERRIDE, ...]) read their own bind-mounted config regardless of host mode, then drop privileges. Zallet can't.Fix
Decouple host uid from container uid the same way the
cookie-permissionssidecar already does for Zebra's RPC cookie — rather than parametrizing the pinned uid, which would force operators to also solve data-volume ownership that distroless Zallet cannot self-heal.scripts/setup-network.sh—chmod 644the copied non-secret TOMLs; grant the age key read to uid 1000 only viasetfacl -m u:1000:r(stays0600for everyone else), with a warning +chmod 644fallback whensetfacl/aclis unavailable.scripts/regtest-init.sh—chmod 644zallet.tomlafter the pwhash rewrite (fixes the mktemp-0600 leak).docs/docker-architecture.md/README.md— document that uid ≠ 1000 operators need no uid coordination for Zallet config; note theacl/setfaclsoft prerequisite and fallback..github/workflows/ci.yaml— reproduce the scripts' permission end-state in regtest staging and assert a--user 1000:1000container can read both config files (guards the mktemp-0600 regression and validates the ACL-through-bind-mount mechanism end-to-end).Note for existing deployments
Re-running the setup scripts won't retro-fix already-present config files (both scripts skip when the live files exist). On an existing box, run:
Verification
bash -npasses on both scripts;ci.yamlparses as valid YAML.🤖 Generated with Claude Code