Add new database versions, Percona Everest support, PMM 3.x fixes - #51
Open
zelmario wants to merge 99 commits into
Open
Add new database versions, Percona Everest support, PMM 3.x fixes#51zelmario wants to merge 99 commits into
zelmario wants to merge 99 commits into
Conversation
- Fork rename: github.com/ihanick/anydbver -> github.com/zelmario/anydbver - Add PSMDB 8.0: 8.0.4-1, 8.0.4-2, 8.0.8-3, 8.0.12-4, 8.0.16-5, 8.0.17-6 - Add PSMDB 7.0: 7.0.24-13, 7.0.26-14, 7.0.28-15 - Add PSMDB 6.0: 6.0.27-21 - Add PBM: 2.11.0-1, 2.12.0-1 - Add PMM Client: 2.43.0-6, 2.43.1-6, 2.43.2-6, 2.44.0-6, 2.44.1-6 - Add PPG 17: 17.5-3, 17.6-1, 17.7-1, 17.7-2 - Add PPG 16: 16.10-1, 16.11-1, 16.11-2 - Add PPG 15: 15.14-1, 15.15-1, 15.15-2 - Add PPG 14: 14.19-1, 14.20-1, 14.20-2 - Add PPG 13: 13.22-1, 13.23-1, 13.23-2
- Add PPG 18 for el8, el9, el10 (x86_64 and aarch64) - Add PPG 17 aarch64 entries for el8, el9, el10 - Add PPG 13-16 el10 entries (x86_64 and aarch64) - Add PPG 12-16 aarch64 entries for el8, el9 Total 151 new entries covering all available Percona PostgreSQL packages from the Percona repositories.
The project migrated to Go; these files are no longer referenced by any active code. Removes 8,319 lines of legacy code including the old Python CLI, Bash wrapper, LXD/Podman controllers, legacy test scripts, and the unmodified_docker Python package.
- Add PXC 8.4.0-8.4.7 version data for el8/el9 in SQLite database - Change default PXC version from 8.0 to 8.4 - Add pxc84 soft variable routing in playbook.yml - Add pxc84 Debian package descriptions for stretch/buster/bionic/focal - Fix mysql_native_password incompatibility with MySQL 8.4: create root@'%' with default auth plugin instead of deprecated mysql_native_password which is disabled by default in 8.4 - Mount tools/, library/, configs/ from project root into ansible container so local changes take effect during deployment - Fix project root detection when binary runs from tools/ subdirectory - Decouple Docker image RELEASE_VERSION from binary version - Fix Makefile to derive version from git tags dynamically - Improve galera_join: add timeouts, logging, and proper Ansible failure reporting instead of silent infinite loops
- Fix PMM client package name (remove .x86_64 suffix) - Fix PMM3 repo enable condition (startswith 3 instead of 3.) - Add PMM URL auto-construction with correct password and port - Add disable_gpg_check for PBM installation - Change Percona repo URLs from HTTP to HTTPS - Fix pipe read error with sync.WaitGroup in runner.go
Auto-detect PMM container internal port (8443 for PMM 3, 443 for PMM 2) when no explicit port is specified in server= parameter. Add timeout to PMM server curl health check loop to prevent infinite hang. Print last Ansible task name on timeout for better error diagnostics.
…andling - Fix operator precedence bug in setup.yaml causing psql to hang on EL waiting for password input (task ran on all distros instead of Debian only) - Add timeout parameter to RunPipe for per-caller control (600s for Ansible playbooks, 300s default for other commands) - Fix setup_pmm.sh systemd service name for PPG 14+ using pg_systemd_svc
Replace || true with a retry loop (5 attempts, 5s delay) so transient PMM server API readiness issues are handled instead of masked.
- Fix post_init script to use localhost for pgpass authentication - Add wait for primary cluster before starting standby cluster - Add pg_hba rule for cross-network replication - Add etcd startup retry logic - Add PMM monitoring setup for Patroni-managed PostgreSQL clusters - Support cluster labels in PMM for proper grouping
Add `anydbver everest` command that deploys a local Database-as-a-Service platform powered by Percona Everest on a k3d Kubernetes cluster. How it works: - Pre-flight checks verify docker (running), kubectl, and k3d are installed - Creates a k3d Kubernetes cluster with configurable number of nodes - Installs Percona Everest via Helm from the official Percona Helm charts - Starts a kubectl port-forward to expose the Everest UI on localhost:8080 - Prints access instructions with username and how to retrieve the password Usage: anydbver everest # Deploy with defaults (3 nodes, latest version) anydbver everest --nodes 5 # Deploy with 5 nodes anydbver everest --version 1.4.0 # Deploy a specific Everest version The Everest UI provides a web-based interface to deploy and manage MySQL, PostgreSQL, and MongoDB database clusters on Kubernetes.
When anydbver is executed from outside the repo directory (e.g. via a symlink in ~/.local/bin/), the existing projectRoot detection based on cwd fails and the Ansible container uses stale baked-in playbooks. Fall back to os.Executable() to locate the repo from the binary path.
Deploy PgBouncer as a standalone connection pooler node using the same pattern as haproxy-pg: anydbver deploy pgbouncer:node1 node1 pg Listens on port 6432 with transaction pool mode, proxying to the first listed PostgreSQL backend.
- Add haproxy-patroni keyword with Patroni REST API health checks (port 8008) for proper read-write (/primary) and read-only (/replica) routing - Integrate pgbackrest with Patroni: auto-detect pgbackrest and use it for archive/restore commands in Patroni bootstrap config - Skip pgbackrest systemd restart and archive_command when Patroni manages PG - Create pgbackrest stanza after Patroni bootstrap on leader node - Make restore_command fail gracefully on nodes without pgbackrest - Improve PMM+Patroni setup with better readiness checks and replica handling - Add test case and help example for full HA stack deployment
…pping) REPLACE INTO mariadb_version used positional VALUES with only 9 values for a 13-column table, so 'anydbver update mariadb' aborted. The mapping was wrong too: the packages string landed in cnf_file. Spell out the column list in a shared mariadbInsertQuery const and add a mysqlCnfFile() helper matching the percona_server_version convention (EL -> /etc/my.cnf.d/zz_mysqld.cnf, Debian family -> /etc/mysql/my.cnf). Note the resulting Debian rows are not deployable yet: roles/mariadb still hardcodes its apt repo to 10.5/focal and installs from static YAML rather than the version DB.
…the skill - general_version anydbver row 0.1.37 -> 0.1.38 (the startup nag compares it to the binary's ldflags version) and IMAGE_VERSION in images-build/build.sh - skill references: k8s-cnpg in products.md, its options in options.md, three worked examples in examples.md, verified dates bumped - one new eval covering the CNPG single-instance case
Adds os:sles15 (aliases sles, sles-15, sles15.7, suse, suse15) so a real SUSE node can be brought up for hand testing. The image is built from registry.suse.com/bci/bci-base:15.7, whose SLE_BCI repository is freely redistributable, so no SUSE subscription is needed to build or run it. The result is a genuine SLES 15 SP7 userland with systemd, sshd and a working zypper. No software can be installed on it. Nobody publishes zypper repositories anydbver can consume yet -- repo.percona.com serves only apt/ and yum/ -- so the version database has no sles15 rows. Deploying a product on a SUSE node used to die inside a role with "'list object' has no attribute 'packages'" once the version lookup came back empty; a guard in playbook.yml now fails on the first task naming the products involved and what to do instead. Keep node_software_requested in sync when a new node-level keyword is added. dist detection needed the SUSE arms: ansible reports ansible_distribution_release '7' on SLES 15.7, so the old fall-through silently produced dist=7. It now yields sles15/sles16, and ansible already exposes ansible_os_family Suse and ansible_pkg_mgr zypper for whenever the install paths grow a third package manager. The base image is published under RELEASE_VERSION (0.1.23) like the other eight, so existing images do not have to be re-pulled. Verified: bare os:sles15 and os:sles deploy clean (failed=0), two SUSE nodes at once, a mixed sles15 + el9 + jammy namespace, the guard firing on os:sles15 pg:17, and el9 + pg:17 still deploying with PostgreSQL running.
`k8s-crunchy` / `crunchy` deploys upstream PGO on k3d through the same flow as the Percona operators, so no Go changes were needed. PGO ships the two files that flow needs under different names: config/default is a kustomize overlay rendered into deploy/bundle.yaml, and examples/postgrescluster is copied to deploy/cr.yaml. Default version 5.8.8, any published version deploys (5.7.9, 5.8.5 to 5.8.8, 6.0.0 to 6.0.2). Options: replicas, db-version, storage, memory, cluster-name, namespace, sql. Traps handled: - Crunchy tags the repository before it publishes the images, so the newest tag is not deployable. The operator image is checked in the registry first, with a fallback to the tag the checkout ships and a clear error otherwise. - config/default at tag v5.8.8 pins ubi9-5.8.6-0, so the version is rewritten into the tag, keeping the distro prefix (ubi8 on 5.7, ubi9 from 5.8 on). - The Patroni leader keeps role=master upstream, not the role=primary the Percona fork switched to in 2.6.0. - The rendered install carries no Namespace object, so postgres-operator is created before applying it. - The label wait returns at 1 of N instances, so it waits on readyReplicas too. - db-version is validated against the operator's RELATED_IMAGE_POSTGRES_<n>. PMM and MinIO are skipped with a warning, same as CloudNativePG. Verified on k3d: 5.8.8 with 3 instances on PostgreSQL 18.1 and both replicas streaming, and 6.0.2 with replicas=1,db-version=17,storage=2Gi on PostgreSQL 17.10. sql= loads, destroy leaves nothing behind. The ansible image must be rebuilt before release, tools/ changed.
`k8s-crunchy` gains two options. Deploying `k8s-minio` alongside it converts pgBackRest repo1 to an S3 repository and leaves repo2 on a local volume. PGO writes its replica-create backup to repo1, so the bucket fills as soon as the cluster comes up. `bucket=`, `region=` and `s3=https://KEY:SECRET@host:port/bucket` retarget it at a real S3 or an external MinIO. Percona's PG operator is a fork of Crunchy PGO, so the CR fields are the ones enable_pgv2_s3 already patches. `expose` puts `<cluster>-ha`, the service Patroni points at the leader, on a LoadBalancer and gives `-replicas` and `-pgbouncer` NodePorts. `<cluster>-primary` is headless and cannot be exposed at all. Traps handled: - pgBackRest speaks HTTPS to S3 and nothing else. A plain http endpoint fails the stanza with "expected protocol 'https' in URL" and takes the local volume repository down with it, so MinIO without TLS is skipped with a warning that names the fix, and an explicit `s3=http://...` is rejected. - Only one service can own host port 5432. k3s servicelb runs a DaemonSet per LoadBalancer that binds the port on every node, so three LoadBalancers leave two of them Pending for ever. - minio_tls_enabled is extracted from run_minio_server because the operator run is a separate process and has to reach the same conclusion about the endpoint. Two pre-existing bugs fixed on the way: - prepare_operator_repository died with "would clobber existing tag" once Crunchy re-pointed v5.8.9 and v6.0.3. The fetch now passes --force. This affects every operator, not just Crunchy. - The operator step's exit code was swallowed by the trailing chown, so a Python traceback still reported success. The status is saved and re-raised. Also documents `sql=`, which was missing from the operator options table since CloudNativePG landed. Verified on k3d: repo1 on MinIO over HTTPS with a full backup Completed and 1289 objects in the bucket, psql through all three endpoints with replicas=2, and the no-cert-manager path degrading to volume repositories. Destroy clean. The ansible image must be rebuilt before release, tools/ changed.
193 new rows. Each was generated from the newest existing row for that major and OS/arch, then every package name was checked against the repository listing rather than trusted, which caught three things a plain version substitution would have got wrong: percona-mysql-shell has no 8.4.11 build and stays at 8.4.10-1, percona-server-mongodb-debuginfo was split into one debuginfo package per component, and PGDG now stamps the RHEL minor into the file name. pg 18.6, plus 14.24, 15.19, 16.15 and 17.11 on RPM and apt ppg 18.6, plus every missing 14, 15, 16 and 17 build ps 8.4.11-11.1, 8.0.46-37.1 pxc 8.4.10-10.1, 8.0.46-38.1 psmdb 8.3.7 (new series, psmdb-83 repo) plus 8.0.29, 7.0.40, 6.0.29 mysql 26.7.0 and the 9.7 LTS line mariadb 12.3.2 on el8 and el9 pxb 8.0.35-36.1 pmm-client 3.9.1-1 proxysql 2.7.3-1.2 k8s-pg 3.0.0 pbm 2.15.0, pxb 8.4.0-6.1, PMM Server 3.9.1 and the psmdb, ps, pxc and cnpg operators were already current. k8s-crunchy 6.0.3 is tagged but its image is not published, and ps and pxb 9.7.1 have Docker images but no yum repository yet, so none of them are included. Every MySQL 9.x row was undeployable. Oracle ships each release stream in its own repository and the release rpm leaves all but 8.0 disabled, while the role only enabled the innovation repo for 8.1 to 8.3, so asking for 9.6.0 silently installed 8.0. The version row now names the repository to enable and the role acts on it: 9.0 to 9.6 and 26.x come from mysql-innovation-community, and 9.7 is LTS with its own release rpm that needs no switch. Smoke tested on el9: psmdb 8.3.7-1, ps 8.4.11-11, mysql 9.7.2 and mysql 26.7.0 all install and run, failed=0. The rest are format verified against the repositories. k8s-pg 3.0.0 is not deploy tested; its deploy/ layout and CR apiVersion match 2.9.0. roles/ changed, so the ansible image must be rebuilt before release.
ANSIBLE_VERSION, IMAGE_VERSION and the general_version anydbver row all move together. The SQL row is what ReadDatabaseVersion returns, so leaving it behind makes every up-to-date user see the version-database nag on every command.
psmdb:latest is 8.3 now rather than 8.0, mysql:latest is 26.7 after MySQL moved to calendar versioning, and mariadb:latest is 12.3. Records how to pin the previous line for each.
Docker Desktop keeps rewriting credsStore=desktop.exe into the WSL2 docker config. The helper is a Windows binary, so the push dies with "exec format error" long after the build finished. QEMU binfmt does not survive a WSL2 restart either, and without it a cross-arch build produces the wrong image instead of failing. Both have cost a release each; check them up front and print the fix. ANYDBVER_SKIP_PREFLIGHT=1 bypasses.
The README never linked the reference documentation, so 1125 lines of instructions.md were unreachable from the front page. The keyword table had also gone stale: barman, haproxy-patroni, kerberos, minio, mydb, nfs-client, nfs-server, pgbouncer, k8s-pmm-ha, cloudnative-pg-operator and crunchy-postgres-operator were all missing, some since v0.1.33. It now matches 'anydbver deploy help keywords' exactly, all 42 of them.
Deploying k8s-pg:3.0.0 left a fully working cluster but ended with exit
code 4 after ten minutes of waiting. cluster_labels() gated the
Crunchy-PGO selector behind op_ver.startswith("2"), so 3.x fell through
to the pre-2.0 label name=<cluster>, which no pod carries. The wait loop
then burned COMMAND_TIMEOUT and raised on a healthy cluster.
The same prefix test hid a second bug: replicas= was silently dropped on
3.x, so you got the CR default instead of the requested instance count.
Both sites now compare numerically through pg_op_ver_at_least(), which
also treats main and branch names as the newest series. The 2.6.0
master -> primary rename and the 1.x fallback are unchanged.
Coroot is an open-source alternative to PMM, and until now there was no way
to bring one up from anydbver. This adds the two keywords that mirror the
pmm / pmm-client pair:
anydbver deploy node0 coroot \
node1 psmdb:latest,replica-set=rs0 coroot-client:server=node0 \
node2 psmdb:latest,replica-set=rs0,master=node1 coroot-client:server=node0
coroot selects docker-image mode on its own and brings up five containers:
the UI, plus -clickhouse, -prometheus, -cluster-agent and -node-agent
sidecars. They all join the namespace network, so destroy already removes
them. coroot-client installs nothing on the database node, it reads the type
and credentials from the database keyword next to it and registers the node
over coroot's API after the playbook. user=, password= and port= override.
Three things this had to get right:
The node-agent starts after the playbook, never with the coroot container.
It builds its container list by scanning /proc at startup and never picks up
containers created later, so it has to come up with the databases already
running, and gets bounced with stop/start on a re-deploy. docker restart is
unreliable for a --pid host container.
Coroot needs a machine-id or it builds no nodes, which means no applications
and nowhere to attach a database integration. Real Linux hosts have one, the
Docker Desktop VM does not, and its /etc/machine-id is a symlink to
/var/lib/machine-id: an absolute symlink read through /proc/1/root resolves
against the reading container's root, not the host's, so the agent cannot
follow it. We write the two real files instead. No-op on Linux.
Application ids are probed directly as <project>:_:Unknown:<container>
rather than scanned out of the overview list, which does not always carry an
application that already exists.
Verified live: PSMDB 8.0 three-node replica set (48 mongo metric families,
correct PRIMARY on the master= node), MySQL 26.7.1 and PostgreSQL 16.15 via
docker-image nodes, and a clean destroy.
Binary and SQL only, no ansible image rebuild needed.
A full deploy can print nothing for over five minutes and read as a hang. Measured on a psmdb replica set: 313s of silence after "included: repo_cache.yaml" while packages install, then 173s, then 167s waiting for coroot to discover the nodes. Nothing on screen the whole time. RunPipe now prints a heartbeat when a streamed command goes quiet: ... still working: 2m14s elapsed, quiet for 1m30s, last step: TASK [mongodb : install packages] It fires only after 45s of no output and repeats at that interval, so normal ansible streaming is unaffected. ANYDBVER_PROGRESS_INTERVAL overrides the interval in seconds, 0 turns it off. This covers the playbook, the operator run, k3d cluster creation and the helm install, since they all stream through RunPipe. The cache image messages were also misleading. "Building cache image ..." was printed before checking that Dockerfile.anydbver.cache exists, so a deploy with caching disabled claimed to build an image, silently did not, and then spent twelve minutes installing packages. The message now appears where the build actually starts and says it will take minutes, and the missing Dockerfile case says so instead of returning silently. coroot's own wait reports which node it is waiting for, every 20s.
--keep is documented as "do not remove existing containers and network", but it only skipped the implicit destroy. Everything after it still tried to create the whole environment from scratch, so the first node hit docker: Error response from daemon: Conflict. The container name "/crt-zelmar-node1" is already in use and the deploy died. The flag was unusable for its stated purpose, which is adding a node to a running environment. The flag now reaches the places that create containers, via a Keep field on ContainerConfig for the ansible path and a parameter on deployHost for the docker-image path. A node that already exists is reused: started first if it was stopped, then left alone. The ansible inventory still includes it, so the playbook runs over the whole set as before and configures whatever is new. Products with sidecars, coroot being the one that has them, create them together with the main container, so the main container existing means the node is there and the whole dispatch is skipped. The network was never the problem, createNamespace already ignores "already exists". Verified on a live namespace: re-running the exact command that failed now reports "Reusing running container crt-zelmar-node1 (--keep)" for the ansible nodes and "Reusing existing crt-zelmar-node0 (--keep)" for the coroot node, runs the playbook, and leaves the environment serving metrics.
Coroot's Postgres collector asks for pg_stat_statements on every scrape, and anydbver enables it nowhere, so every PG node logged pq: relation "pg_stat_statements" does not exist (42P01) and the query views stayed empty. The per-query statistics are the closest thing coroot has to PMM's Query Analytics, so this was the one real gap left in the PG support. anydbver already does this for PMM in tools/setup_pmm.sh. coroot-client now does the same from Go, which keeps the feature binary-only with no ansible role change: ALTER SYSTEM, one restart, CREATE EXTENSION. It appends to shared_preload_libraries instead of overwriting it, so a node that already preloads repmgr or pg_stat_monitor keeps them, and it does nothing on a standby, where ALTER SYSTEM is pointless and CREATE EXTENSION fails on a read-only server. A node with no systemd, meaning one deployed from a plain docker image, is skipped. Failure is logged and the deploy continues, since it costs the query views and nothing else. Verified live on ppg:17: shared_preload_libraries went from empty to pg_stat_statements, the 42P01 warnings stopped, and pg_ metric families went from 18 to 25, the new ones being pg_top_query_calls_per_second, pg_top_query_time_per_second and pg_top_query_io_time_per_second with normalised query text.
The UI password was ANYDBVER_DEFAULT_PASSWORD, which is awkward to type into a browser for a throwaway local dashboard. It is admin/admin now, matching what k8s-pmm-ha already defaults to. Both the bootstrap and the API client read the same constant, so they cannot drift. Also fixes a case the --keep work left open. deployHost decides a docker-image node exists by looking at the main container, so a coroot node whose server container was removed while its sidecars survived went on to create them again and failed on the name. CreateCorootContainer only ever runs when the coroot node itself is absent, so any sidecar still holding one of its names is a leftover of the node being rebuilt, and is now removed first. Verified live by deleting only crt-zelmar-node0 and redeploying with --keep: the four orphaned sidecars were replaced without a conflict, the databases re-registered, admin/admin returns 200 and the old password 404.
The coroot section had three examples and lived only in instructions.md. A new keyword needs to reach four places or people will not find it, so: - instructions.md: an example per case — MongoDB replica set and single node, PostgreSQL primary+replica and vanilla PG, Percona Server and PXC, Valkey, databases from plain docker images, several engines under one coroot, PMM and coroot side by side on the same database, and adding a database to a running coroot with --keep. Plus the credential override and the note that registration takes a minute or two rather than hanging. - README.md: both keywords in the list, one example in the keywords block. - The anydbver skill: a Coroot section in references/examples.md, the two keywords in references/products.md, their options in references/options.md, a playbook entry and three gotchas in SKILL.md, and coroot in the skill description so it triggers. Every example is a shape that was run for real, except pxc and the PMM-alongside-coroot one, which are structurally identical to tested shapes.
Bumps the general_version anydbver row to 0.1.40 so the "version database update is available" nag stays quiet for up-to-date users, and commits the three ldflags-stamped binaries. ANSIBLE_VERSION deliberately stays at 0.1.39: nothing in this release touches roles/, common/, tools/ scripts, configs/ or playbook.yml, so the 0.1.39 ansible image is still correct and there is no new tag to point at.
A MySQL node was registered and scraped but did not appear in coroot's
overview, which reads as the integration having failed. It had not.
Coroot's overview skips applications it considers standalone
(api/views/overview/applications.go), and IsStandalone means no observed
traffic to or from any other application. A single database nothing has
connected to is therefore hidden. Coroot learns those connections from the
node-agent's eBPF tracking, and on Docker Desktop the cluster-agent's own
connections open after the agent starts, so they are never recorded. A
replica set is unaffected: its members talk to each other from processes the
agent already knew about.
Nothing to fix in the pipeline, so the deploy now hands over a direct link
per database instead:
Monitored databases:
node1 mysql http://127.0.0.1:9080/p/<proj>/applications/<app-id>
The application id has to be percent-encoded, colons included, or coroot
answers "invalid application id" — the first version of this printed the raw
id and did exactly that.
Documented in instructions.md and in the skill gotchas.
Tearing down a namespace that ran coroot could fail: cannot remove container "b028ceff1fac": could not kill container: container ... PID 1378947 is zombie and can not be killed Under Docker Desktop a privileged --pid host container, which is what the coroot node-agent is, can end up as a zombie that docker rm -f cannot kill. docker rm removes the rest and fails only on that one, so the namespace is left half torn down with its network still in use and destroy exits as if it had worked. Recovering by hand meant a second docker rm -f once the container had reached Exited, plus removing the network. destroy now re-lists whatever is still attached to the namespace network after the bulk removal, stops it and removes it again. Nothing is left to survive, so the following network rm succeeds. The normal path is unchanged: with nothing left the extra listing is one docker ps and no output. Verified by racing a container onto the namespace network mid-destroy, which reproduces the "bulk removal missed something" shape deterministically: the new pass reported "1 container(s) survived removal, stopping them first", removed it, and the network came away clean. The zombie itself is intermittent and was not reproduced on demand, but the recovery sequence is the one that worked by hand.
A monitored database never showed up in coroot's application list. It was registered and scraped, its metrics were in Prometheus, but the list only ever had coroot's own containers in it. Coroot skips applications where IsStandalone() holds, meaning it has seen no traffic between that application and any other. It learns those links from the node-agent, and the node-agent records a connection only if the connection exists while it scans. The ordering made that impossible: the node-agent was started right after the playbook, then the databases were registered, and only then did the cluster-agent start connecting to them. By the time there was any traffic to observe, the agent had already scanned. So the deploy now restarts the node-agent a second time, after waiting for the cluster-agent to report its first completed collection. The cluster-agent also polls coroot for config every 15s instead of the default 60s, which shortens that wait. Verified on a clean deploy of coroot + ps:latest: the overview went from 3 applications to 5, with mysqltest-zelmar-node1 listed as type=mysql, stable over several minutes of polling. Coroot still needs a few refresh cycles after the deploy exits, so the printed links and the docs now say the list takes a couple of minutes to catch up rather than claiming a lone database stays hidden forever.
The v0.1.40 assets were replaced in place rather than cutting v0.1.41, so the announcement already sent stays valid. Binaries now carry: 3195a0d coroot: restart the node-agent once the cluster-agent is scraping 52bacf6 destroy: pick up containers that survive the bulk removal The first is the important one: without it a coroot-monitored database never appeared in coroot's application list, which is the first thing anyone following the announcement would try. The v0.1.40 tag still points at a3f072a, so these binaries are ahead of the tag by two commits. The release notes say so, and each binary reports its own commit via --version.
"anydbver deploy ps:9.7" ran for six and a half minutes installing base
packages and then died with
The task includes an option with an undefined variable ...
'dict object' has no attribute ''
Two separate problems behind that.
The version really exists. Percona Server 9.7.1-1 is published as an LTS
series in the ps-97-lts repo for el8, el9 and el10 on both arches, and
anydbver's database simply had no 9.x rows: percona_server_version is
maintained by hand, and the updater only knows postgresql, mariadb and
pmm-client, so nothing was ever going to add them. Six rows added,
generated from the actual repo listings rather than by substituting version
strings, which is how the missing el8 percona-server-debuginfo package was
caught before it became an empty entry in debug_packages.
The error was also unreadable. The role's guarded lookup falls back to
os[dist][soft] when the version query returns nothing, and `soft` is derived
from a list of version prefixes the playbook knows about, so an unknown
version leaves it empty and Jinja complains about a dict with no attribute.
deploy now checks every requested version against the version database
before creating anything, in the same place unknown keywords are already
rejected, and says what is available:
Error: Percona Server for MySQL 9.99 is not in the version database (from "ps:9.99").
Available: 9.7.1, 8.4.11, 8.4.10, 8.4.8, ...
Run 'anydbver versions ps' for the full list, or 'anydbver update' to refresh it.
Versions taken from a docker image tag, "latest", and software with no
version_sources row are left alone. Data plus Go only, no role change.
Verified: ps:9.7 deploys clean (failed=0) and the server reports
9.7.1-1 Percona Server (GPL) from percona-ps-97-lts-release.repo.
Swept every deployable product against its upstream repository, numerically rather than with sort(1), which lies about 8.4.8 versus 8.4.10. Three were behind: - psmdb 8.3.8-2 (had 8.3.7-1), four rows, el8/el9 x86_64 and aarch64 - mariadb 12.3.3-1 (had 12.3.2-1), two rows, el8/el9 x86_64 - CrunchyData/postgres-operator 6.0.3. Its git tag was already there at v0.1.39 but the image 404'd, which is the standing Crunchy trap. The ubi9-6.0.3-0 image is published now, confirmed with docker manifest inspect, so the row is safe to add. Everything else is current: ps (after 9.7.1), pxc, pbm, ppg, pg, mysql, pmm-client, pmm-server, and the four Percona operators plus CloudNativePG. pxb is up to date on both maintained lines, 8.0.35-36.1 and 8.4.0-6.1. It was worth re-checking because ps 9.7 turned out to exist after being held back, but pxb 9.7 is still only published as 9.7.1-1.rc1, a release candidate, so it stays out. Rows generated from the repository listings, not by substituting version strings. Smoke tested psmdb:8.3, which resolves to 8.3.8 and deploys failed=0; the rest are format verified against their templates.
…resVersion
"anydbver deploy k3d k8s-pg:3.0.0" never came up. The operator, pgBouncer and
the pgBackRest repo host ran fine; all three Postgres instances sat in
Init:CrashLoopBackOff, so the deploy waited for a primary that could never be
ready.
Operator 3.0.0 changed how images are referenced. 2.x tags every component on
the operator repository:
percona-postgresql-operator:2.6.0-ppg16.8-postgres
3.0.0 uses separate repositories with their own versions:
percona-distribution-postgresql:18.3-2
percona-pgbouncer:1.25.1-1
percona-pgbackrest:2.58.0-1
anydbver rewrites the "ppg<major>-<component>" form to deploy a different PG
major. Against a 3.x cr.yaml that pattern matches nothing, so the substitution
was a silent no-op and the images stayed on PG 18.3, while the next step still
set spec.postgresVersion=17. The operator refuses that combination and the
instance pods crashloop.
It fired on a default deploy because --db-version=17 was injected
unconditionally: the k8s_arguments row was always_add with no version filter.
Nobody asked for 17.
Two changes.
The db-version row is now split by operator major. 2.x keeps the old default
of 17. 3.x only passes db-version when it is asked for, so a plain
"k8s-pg:3.0.0" deploys the major the operator ships with and nothing is
rewritten. This is data only.
set_pg_major_in_images() in tools/run_k8s_operator.py handles both schemes, so
"k8s-pg:3.0.0,db-version=17" works too: it resolves the major to a published
percona-distribution-postgresql tag (17 -> 17.11.1-1) and retags only the
postgres image, leaving pgbouncer and pgbackrest alone since their versions
are no longer tied to the PG major. If no image exists for the requested
major it says so and leaves spec.postgresVersion alone rather than recreating
the mismatch.
Verified against the real cr.yaml of both operators: 3.0.0 gets
percona-distribution-postgresql:17.11.1-1 with the other images untouched,
2.6.0 still gets 2.6.0-ppg15-{postgres,pgbouncer,pgbackrest}. Arg generation
checked for all four combinations of operator major and explicit db-version.
run_k8s_operator.py is baked into the ansible image, so the second half needs
an image rebuild to reach users; the first half does not.
"anydbver deploy mariadb:10.4" failed with Failed to download metadata for repo 'mariadb': Cannot download repomd.xml Three separate faults, none of them specific to 10.4. yum.mariadb.org only carries maintained series. 14 of the 19 series we ship rows for have been removed as they went EOL: 10.1 10.2 10.3 10.4 10.5 10.7 10.8 10.9 10.10 11.1 11.2 11.3 11.5 11.6. Only 10.6, 10.11, 11.4, 11.8 and 12.3 are still there. Both the package URLs and the generated MariaDB.repo pointed at that mirror, so most of the table was dead, and dnf refuses to install anything at all when an enabled repo cannot be refreshed, even though the packages are addressed by full URL. archive.mariadb.org keeps every release for ever, current ones included, so all 444 rows and the repo file now point there. That also removes the need to track which series are still maintained. The live mirror additionally only keeps each series' newest patch, which is why an older 10.11 or 11.4 patch could not be found on it either. MariaDB has never worked on Apple Silicon. Every row in the table was x86_64, and the lookup in roles/mariadb/vars/main.yml filtered on os but not arch, unlike every other role, so an arm64 host was silently handed amd64 packages and failed later with a confusing error. MariaDB does publish aarch64, so there are now 193 aarch64 rows, the lookup filters on ansible_architecture like the others do, and the repo baseurl is no longer hardcoded to -amd64. Rows generated from the archive listings, not by substituting version strings, which is how the el7 ".centos" infix (10.1.39-1.el7.centos.x86_64) was picked up. x86_64 coverage goes from 252 combinations to 251: 10.9.1 el7 is dropped because MariaDB never published RPMs for it, so that row could never have deployed. Verified live: mariadb:10.4 now installs 10.4.34 from archive.mariadb.org/mariadb-10.4.34/yum/rhel8-amd64 and the server reports 10.4.34-MariaDB-log. The aarch64 side is verified by listing rather than by deploy, since this host is x86_64. roles/ changed, so this needs the ansible image rebuild to reach users.
Bumps ANSIBLE_VERSION and IMAGE_VERSION to 0.1.41 and the general_version anydbver row to match, and commits the three ldflags-stamped binaries. Unlike v0.1.40 this one does need new ansible images: roles/mariadb and tools/run_k8s_operator.py changed. Both zelmar/rockylinux:8-anydbver-ansible-0.1.41 and -0.1.41-arm64 are built and pushed. Verified with the release binary against the published image: mariadb:10.4 installs 10.4.34 from archive.mariadb.org and the server reports 10.4.34-MariaDB-log.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
New features & version support
Add Percona PostgreSQL 18 and missing PPG versions
Add PXC 8.4 support and fix MySQL 8.4 compatibility issues
Add Percona Everest deployment support
Add PMM 3.6.0 server and client version support
Change the default PSMDB version from 7.0 to 8.0
PMM 3.x fixes
Fix PMM client 3 installation and various deployment issues
Fix PMM client mongo setup to support version 3
Fix PMM client setup for PMM 3.x docker-image deployments
Infrastructure & maintenance
Fix Patroni cluster setup
Add less and nano to all container base images
Remove deprecated Python/Bash implementation and legacy files
Change Docker image publisher from ihanick to zelmar
Rebuild binary with version info