[feature] Wire persistent-mass-upgrade Beat tasks into docker stack #623 by Eeshu-Yadav · Pull Request #628 · openwisp/docker-openwisp

Eeshu-Yadav · 2026-06-02T07:22:21Z

Checklist

I have read the OpenWISP Contributing Guidelines.
I have manually tested the changes proposed in this pull request.
I have written new test cases for new code and/or updated existing tests for changes to existing code.
I have updated the documentation.

Reference to Existing Issue

Closes #623.

Related to openwisp/openwisp-firmware-upgrader#379 (parent — Persistent Mass Upgrades) and openwisp/openwisp-firmware-upgrader#436 (upstream implementation).

Description of Changes

Once openwisp-firmware-upgrader ships the persistent-mass-upgrades feature, a docker install still won't fire its retry loop or its reminder notifications. The new behaviour rides on two Celery Beat tasks (check_pending_upgrades and send_pending_upgrade_reminders), and images/common/openwisp/celery.py merges per-module Beat dicts but doesn't have a firmware one. Only the task routes are wired up. Operators would install the new release and quietly get none of the new behaviour.

This PR adds the missing schedule.

A firmware_schedule block lands alongside the existing monitoring_schedule, radius_schedule, etc. Gated on the same USE_OPENWISP_FIRMWARE + USE_OPENWISP_CELERY_FIRMWARE envs that gate the firmware task routes today, so disabling either env hides both the routes and the schedule.
Two new env vars let operators tune cadence without rebuilding: OPENWISP_FIRMWARE_CHECK_PENDING_PERIOD_MINUTES (default 10, the retry-scan period) and OPENWISP_FIRMWARE_REMINDER_SCAN_PERIOD_DAYS (default 7, how often the reminder task looks at long-running batches). The per-batch reminder cadence stays controlled upstream by OPENWISP_FIRMWARE_UPGRADER_PERSISTENT_REMINDER_PERIOD, default 60 days.
test_celery in tests/runtests.py now expects three more registered tasks: check_pending_upgrades, retry_pending_upgrade, send_pending_upgrade_reminders. If a future change drops the wiring, CI flags it instead of leaving the feature silently dead on prod.
Settings doc entries added next to OPENWISP_CELERY_FIRMWARE_COMMAND_FLAGS.

The second commit ([chores] TEMPORARY pin openwisp-firmware-upgrader to gsoc26 branch) is what it says on the tin: flips OPENWISP_FIRMWARE_SOURCE to the upstream gsoc26 branch so CI on this integration branch installs a version that actually ships the new tasks. Must be reverted to a released-version pin as the final commit before this branch merges to master.

coderabbitai · 2026-06-02T07:22:29Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 2dbda593-436f-4c2f-b3cf-344b74a48680

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch gsoc26/wire-persistent-upgrade-beat-tasks

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Copilot

Pull request overview

This PR wires the new persistent mass-upgrade Celery Beat tasks from openwisp-firmware-upgrader into the docker-openwisp Celery app, exposing operator-tunable scheduling via environment variables and extending the Celery task registration regression test to cover the new tasks.

Changes:

Add firmware-related Beat schedule entries (check_pending_upgrades, send_pending_upgrade_reminders) gated behind existing firmware enable flags.
Expose scheduling cadence via OPENWISP_FIRMWARE_CHECK_PENDING_PERIOD_MINUTES and OPENWISP_FIRMWARE_REMINDER_SCAN_PERIOD_DAYS in the base image, .env, and documentation.
Extend tests/runtests.py::test_celery to assert the new firmware upgrader tasks are registered.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
`images/common/openwisp/celery.py`	Adds firmware Beat schedule entries and merges them into `beat_schedule`.
`images/openwisp_base/Dockerfile`	Exposes new firmware schedule env defaults; temporarily pins firmware-upgrader source.
`tests/runtests.py`	Updates Celery task registration assertions to include new firmware tasks.
`docs/user/settings.rst`	Documents the two new firmware scheduling environment variables.
`.env`	Adds defaults for the new firmware scheduling environment variables.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

-ARG OPENWISP_FIRMWARE_SOURCE="openwisp-firmware-upgrader~=1.2.0"
+ARG OPENWISP_FIRMWARE_SOURCE="openwisp-firmware-upgrader @ git+https://github.com/openwisp/openwisp-firmware-upgrader.git@gsoc26-persistent-scheduled-upgrades"
 # hadolint ignore=DL3013
 RUN pip install --no-cache-dir --user --upgrade ${OPENWISP_FIRMWARE_SOURCE}


 # hadolint ignore=DL3013
 RUN pip install --no-cache-dir --user --upgrade ${OPENWISP_MONITORING_SOURCE}
-ARG OPENWISP_FIRMWARE_SOURCE="openwisp-firmware-upgrader~=1.2.0"
+ARG OPENWISP_FIRMWARE_SOURCE="openwisp-firmware-upgrader @ git+https://github.com/openwisp/openwisp-firmware-upgrader.git@gsoc26-persistent-scheduled-upgrades"


+    check_pending_minutes = int(
+        os.environ.get("OPENWISP_FIRMWARE_CHECK_PENDING_PERIOD_MINUTES", "10")
+    )
+    reminder_scan_days = int(
+        os.environ.get("OPENWISP_FIRMWARE_REMINDER_SCAN_PERIOD_DAYS", "7")
+    )


openwisp-companion · 2026-06-02T07:26:40Z

Docker Build Failure and Doc Formatting Error

Hello @Eeshu-Yadav,
(Analysis for commit 5e9803a)

Build Failure: The openwisp-firmware-upgrader installation failed due to an invalid package name format in the Dockerfile. The @ symbol is not a valid part of a package name when installing from a Git repository.

Fix: Remove the @ symbol from the OPENWISP_FIRMWARE_SOURCE argument in the images/openwisp_base/Dockerfile on line 47. It should be openwisp-firmware-upgrader git+https://github.com/openwisp/openwisp-firmware-upgrader.git@gsoc26-persistent-scheduled-upgrades.

Documentation Formatting: The docs/user/settings.rst file has formatting issues that were detected by the ReStructuredText check.

Fix: Please reformat the docs/user/settings.rst file. You can typically fix this by running a documentation formatting tool or by manually adjusting the line breaks and spacing to adhere to ReStructuredText conventions.

openwisp-companion · 2026-06-02T07:39:40Z

The CI is failing due to transient infrastructure issues (not related to your code). I have restarted the failed jobs automatically (1/3).

openwisp-companion · 2026-06-02T07:44:14Z

The CI is failing due to transient infrastructure issues (not related to your code). I have restarted the failed jobs automatically (2/3).

openwisp-companion · 2026-06-02T07:48:25Z

The CI is failing due to transient infrastructure issues (not related to your code). I have restarted the failed jobs automatically (3/3).

openwisp-companion · 2026-06-02T07:52:59Z

Docker Build Failure: Git Command Not Found

Hello @Eeshu-Yadav,
(Analysis for commit 2d65990)

The CI build failed because the Docker image build process could not find the git command. This is required to install the openwisp-firmware-upgrader package from source.

Fix:

Ensure that git is installed in the openwisp_base Docker image. You can do this by adding git to the apt-get install command in the images/openwisp_base/Dockerfile.

Specifically, modify the RUN apt-get update && apt-get install ... command to include git.

…ders into the Beat schedule #623 Added a firmware_schedule alongside the existing per-module schedules in images/common/openwisp/celery.py, gated on USE_OPENWISP_FIRMWARE and USE_OPENWISP_CELERY_FIRMWARE. Cadences read from two new env vars, OPENWISP_FIRMWARE_CHECK_PENDING_PERIOD_MINUTES (default 10) and OPENWISP_FIRMWARE_REMINDER_SCAN_PERIOD_DAYS (default 7), exposed in .env and the openwisp_base Dockerfile. Documented both env vars in docs/user/settings.rst. Extended the test_celery assertion in tests/runtests.py with the three new task names (check_pending_upgrades, retry_pending_upgrade, send_pending_upgrade_reminders). Related to #623

openwisp-companion · 2026-06-02T08:20:52Z

The CI is failing due to transient infrastructure issues (not related to your code). I have restarted the failed jobs automatically (1/3).

openwisp-companion · 2026-06-02T08:27:23Z

The CI is failing due to transient infrastructure issues (not related to your code). I have restarted the failed jobs automatically (2/3).

openwisp-companion · 2026-06-02T08:32:02Z

The CI is failing due to transient infrastructure issues (not related to your code). I have restarted the failed jobs automatically (3/3).

… revert before merge CI on this integration branch installs openwisp-firmware-upgrader~=1.2.0 from PyPI, which does not yet ship the three new tasks. Without this pin the new Beat entries and the test_celery assertion both fail. Points at the openwisp-firmware-upgrader branch issues/417-persistence-schema-fields (the PR #436 head), via the tarball URL so pip does not need git in the slim base image. Revert this to openwisp-firmware-upgrader~=1.2.0 (or the next released version) as the final commit before this branch merges to master, once PR #436 has merged and a release is cut. Related to #623

openwisp-companion · 2026-06-02T11:27:02Z

The CI is failing due to transient infrastructure issues (not related to your code). I have restarted the failed jobs automatically (1/3).

openwisp-companion · 2026-06-02T12:11:09Z

The CI is failing due to transient infrastructure issues (not related to your code). I have restarted the failed jobs automatically (2/3).

…ert before merge After rebasing PR #436 (openwisp-firmware-upgrader) onto its master, the firmware-upgrader code now imports openwisp_utils.api.pagination, which only exists in the openwisp 1.3 family. The existing firmware-upgrader tarball pin (54fb1e9) is no longer enough on its own: openwisp-monitoring/topology/radius 1.2 from PyPI pull in openwisp-utils 1.2 transitively, which lacks api.pagination, and CI fails with ModuleNotFoundError during dashboard data.setup(). Switch the conditional --force-reinstall ARGs (utils, users, controller, notifications, ipam) from default to the matching 1.3 branch tarballs so the entire family is consistent. monitoring, topology and radius stay on 1.2 — only the firmware-upgrader's direct dependencies need bumping. Revert these alongside 54fb1e9 once PR #436 has merged and an openwisp-firmware-upgrader release (and its 1.3 family deps) are cut. Related to #436 Related to #623

The two settings entries added in 35a2433 (the explanation paragraph for OPENWISP_FIRMWARE_REMINDER_SCAN_PERIOD_DAYS) wrap to a different width than docstrfmt --line-length 74 expects, which is what openwisp-qa-check runs in CI. Re-wrapped to match. Related to #623

openwisp-companion · 2026-06-04T12:34:03Z

The CI is failing due to transient infrastructure issues (not related to your code). I have restarted the failed jobs automatically (1/3).

openwisp-companion · 2026-06-04T12:36:20Z

The CI is failing due to transient infrastructure issues (not related to your code). I have restarted the failed jobs automatically (1/3).

…623 a95eadf bumped utils and controller to the 1.3 family but left OPENWISP_TOPOLOGY/MONITORING/RADIUS_SOURCE on ~=1.2.0. network-topology 1.2's admin.py imports UUIDAdmin, which openwisp-utils 1.3 removed, so daphne crashed on import during admin autodiscover and CI failed. CI reads the Dockerfile defaults (.build.env is gitignored), so the pin must live here. Pin the whole openwisp 1.3 family to fixed commit SHAs so the build is consistent and reproducible. Temporary for the firmware-upgrader PR compatibility; revert with the rest of the 1.3 pins before merge. Related to #623

openwisp-companion · 2026-06-10T18:29:04Z

The CI is failing due to transient infrastructure issues (not related to your code). I have restarted the failed jobs automatically (1/3).

openwisp-companion · 2026-06-10T18:50:57Z

The CI is failing due to transient infrastructure issues (not related to your code). I have restarted the failed jobs automatically (1/3).

create_ssh_key_template checked for an existing template via config__contains, which does not match the stored JSON config, so the guard never short-circuited and a second bootstrap run tried to recreate the shared "SSH Keys" template. openwisp-controller 1.3 enforces unique shared-template names, so this raised ValidationError, aborted the bootstrap and left uwsgi down (nginx 502 in the Test step). Guard on the template name like the sibling create_* helpers already do. Related to #623

The device-firmware admin inline reverses the "upgrader" namespace for the upgrade-operation cancel button, but the dashboard urlconf only wired the firmware private-storage URLs, not the API URLs that define that namespace. Rendering the device admin page raised NoReverseMatch and returned 500 (nginx then reported 502), failing the Test step. Include the firmware-upgrader API URLs so the namespace is reversible from the dashboard; nginx still routes the actual requests to the API container. Related to #623

I had pinned radius to a 1.3 commit with the rest of the family, but openwisp-radius 1.3 removed the deactivate_expired_users task that docker-openwisp's test_celery expects, failing the Test step. radius 1.2 still ships that task and does not import the openwisp-utils admin symbols that 1.3 removed, so it stays compatible with the pinned utils 1.3. Revert radius to the released 1.2 (the family-pin commit intentionally kept radius on 1.2; only utils/controller/topology/monitoring need the 1.3 line). Related to #623

Copilot AI review requested due to automatic review settings June 2, 2026 07:22

Copilot started reviewing on behalf of Eeshu-Yadav June 2, 2026 07:22 View session

openwisp-companion Bot mentioned this pull request Jun 2, 2026

[chores:gsoc26] Wire openwisp-firmware-upgrader's check_pending_upgrades into the docker-openwisp Beat schedule #623

Open

Copilot AI reviewed Jun 2, 2026

View reviewed changes

Eeshu-Yadav force-pushed the gsoc26/wire-persistent-upgrade-beat-tasks branch from 5e9803a to 2d65990 Compare June 2, 2026 07:35

Eeshu-Yadav force-pushed the gsoc26/wire-persistent-upgrade-beat-tasks branch from 2d65990 to 4b67b61 Compare June 2, 2026 07:56

Eeshu-Yadav force-pushed the gsoc26/wire-persistent-upgrade-beat-tasks branch from 4b67b61 to 99088d0 Compare June 2, 2026 10:43

Eeshu-Yadav force-pushed the gsoc26/wire-persistent-upgrade-beat-tasks branch from 99088d0 to 54fb1e9 Compare June 2, 2026 10:43

Eeshu-Yadav force-pushed the gsoc26/wire-persistent-upgrade-beat-tasks branch from e73e377 to 60a95c2 Compare June 4, 2026 11:50

Eeshu-Yadav added 2 commits June 4, 2026 17:23

Eeshu-Yadav force-pushed the gsoc26/wire-persistent-upgrade-beat-tasks branch from 60a95c2 to 90afffa Compare June 4, 2026 11:54

Eeshu-Yadav force-pushed the gsoc26/wire-persistent-upgrade-beat-tasks branch from c7c994a to b1880cd Compare June 10, 2026 18:11

Eeshu-Yadav added 3 commits June 11, 2026 00:29

Uh oh!

Conversation

Eeshu-Yadav commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Reference to Existing Issue

Description of Changes

Uh oh!

coderabbitai Bot commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

openwisp-companion Bot commented Jun 2, 2026

Docker Build Failure and Doc Formatting Error

Uh oh!

openwisp-companion Bot commented Jun 2, 2026

Uh oh!

openwisp-companion Bot commented Jun 2, 2026

Uh oh!

openwisp-companion Bot commented Jun 2, 2026

Uh oh!

openwisp-companion Bot commented Jun 2, 2026

Docker Build Failure: Git Command Not Found

Uh oh!

openwisp-companion Bot commented Jun 2, 2026

Uh oh!

openwisp-companion Bot commented Jun 2, 2026

Uh oh!

openwisp-companion Bot commented Jun 2, 2026

Uh oh!

openwisp-companion Bot commented Jun 2, 2026

Uh oh!

openwisp-companion Bot commented Jun 2, 2026

Uh oh!

openwisp-companion Bot commented Jun 4, 2026

Uh oh!

openwisp-companion Bot commented Jun 4, 2026

Uh oh!

openwisp-companion Bot commented Jun 10, 2026

Uh oh!

openwisp-companion Bot commented Jun 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Eeshu-Yadav commented Jun 2, 2026 •

edited

Loading

coderabbitai Bot commented Jun 2, 2026 •

edited

Loading