Release workflows by Flix6x · Pull Request #2260 · FlexMeasures/flexmeasures

Flix6x · 2026-07-01T21:05:49Z

Description

Automations to help with releasing.

Add workflow to build and release docker image, which runs, just like the PyPI release workflow, when a GH release is created. A manual approval step is built in via the new dockerhub environment in the repo settings.
Add manually triggered workflow for QA, specifically, running available tutorial scripts and the HEMS script.
Extend the PyPI release workflow with a smoke test.
Add a script to check PRs since the last tag.
Renamed workflows for clarity, and moved the .sh script to the ci package.
Added changelog item in documentation/changelog.rst

The HEMS step of the QA workflow depends on FlexMeasures/flexmeasures-client#216, which has been merged. It:

adds FLEXMEASURES_CLI_CMD/FLEXMEASURES_CLI_CONFIG_DIR env vars so the HEMS example's report-generation step can run against a dockerized flexmeasures (via docker compose exec -T server flexmeasures) instead of requiring a local CLI install, and
fixes a bug where the client treated the sensor-data-upload endpoint's 202 Accepted (async job) response as an error, which silently broke the HEMS tutorial before it ever reached forecasting/scheduling/reporting.

Look & Feel

...

How to test

Testing the workflows may require merging to main first.

Locally validated the full qa-release.yml logic (docker compose stack + tutorials 1-4 + HEMS with report generation) against flexmeasures-client#216's branch: all 6 HEMS simulation steps completed, all data uploads succeeded, all report-generation calls succeeded via the dockerized CLI, ending in "HEMS Tutorial completed successfully!" with zero failures.

Further Improvements

The tutorial scripts remain on the tsc repo, as well as the original release checklist. Not sure if that situation should remain as is.

Related Items

Depends on FlexMeasures/flexmeasures-client#216 (merged).

Signed-off-by: F.N. Claessen <claessen@seita.nl>

Keep it alongside the other CI scripts (ci/setup-postgres.sh etc.) rather than a standalone bin/ directory. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Runs tutorials 1-4 and the HEMS walkthrough against the local docker compose stack, using the runner scripts kept in FlexMeasures/tsc and the HEMS example kept in FlexMeasures/flexmeasures-client (not duplicated here). Manual trigger only, to avoid slowing down every push while still cutting release QA toil. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

The HEMS example's report-generation step shells out to a local flexmeasures CLI, which doesn't work as-is against a dockerized server. Bind-mount examples/HEMS/configs into the server container and point FLEXMEASURES_CLI_CMD/FLEXMEASURES_CLI_CONFIG_DIR (added in FlexMeasures/flexmeasures-client#qa/hems-report-cli-override) at it, so report generation runs for real instead of being silently skipped. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Gate docker-publish.yml behind the dockerhub GitHub environment, mirroring the approval gate already used by pypi-publish.yml. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…nto ci/ Rename build.yml -> docker-build.yml and release.yml -> pypi-publish.yml so filenames reflect what they actually build/publish, and qa-release.yml -> docker-qa.yml to distinguish it from the release-publishing pair. Move generate-dummy-price.sh into ci/, alongside the other CI helper scripts, and update references accordingly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

nhoening

Very good start, I found just a few things for now.

nhoening · 2026-07-02T10:18:13Z

+Release process
+================
+
+This page describes how FlexMeasures releases are cut, and which parts are automated versus manual.


Suggested change

This page describes how FlexMeasures releases are cut, and which parts are automated versus manual.

This page describes how FlexMeasures releases are done, and which parts are automated versus manual.

I have never seen "cut" used like this, is that usual?

Good catch, fixed — changed to "done".

nhoening · 2026-07-02T10:22:17Z

+* ``uv sync --group dev --group test`` and ``uv run poe test``.
+* For MINOR/MAJOR releases, do QA using the Docker Compose stack, and write the accompanying blog post.
+
+  Tutorials 1-4 and the HEMS walkthrough can be run via the manually-triggered ``QA (release)`` GitHub Actions workflow (``.github/workflows/docker-qa.yml``). It spins up the local docker compose stack and runs the tutorial runner scripts from `FlexMeasures/tsc <https://github.com/FlexMeasures/tsc/tree/main/tsc/scripts>`_ and the HEMS example from `FlexMeasures/flexmeasures-client <https://github.com/FlexMeasures/flexmeasures-client/tree/main/examples/HEMS>`_ (both kept in their own repos, not duplicated here). Trigger it from the Actions tab; it does not run automatically. This does not replace manual UI login/graph checks, tutorial 5, or exploratory QA.


We moved the tutorial runner scripts into the FM repo, at documentation/tut/scripts. There is even a script for tutorial 5 there. And RELEASE.md points to those, actually. I suggest using those, probably makes the QA task lighter, as well.

Maybe they should have been deleted from the TSC repo, I'll do that.

Done — switched the QA workflow and docs to use documentation/tut/scripts (including tutorial 5) instead of checking out tsc.

nhoening · 2026-07-02T10:25:48Z

+This lists merged PRs since the last tag (via ``ci/list-merged-prs-since-tag.sh``), pre-formatted as changelog bullets.
+It is read-only: it does not edit or commit anything.
+Review, categorize each entry into *New features* / *Infrastructure / Support* / *Bugfixes*,
+and paste them into ``documentation/changelog.rst`` (and ``documentation/cli/change_log.rst`` / ``documentation/api/change_log.rst`` where relevant).


This should only be done if PR IDs are missing at that point, right?

Maybe the task can even look that up (missing PR IDs) and suggest which need to still be added.

Good idea, done — the script now cross-references PR numbers already linked in the changelog files and only lists the ones that are still missing.

nhoening · 2026-07-02T10:31:33Z

+          push: true
+          tags: lfenergy/flexmeasures:${{ steps.version.outputs.tag }}
+
+      - name: Build and push (latest)


Technically, we don't need to build again, in the RELEASE file we tag the image from the last step as latest, then push that.

I know that Docker will also use the cached layers so the end result is maybe the same, but tagging seems cleaner (pushing the very same image).

Agreed, fixed — now a single build-push-action call with both tags, so the exact same image gets pushed under both, no second build.

nhoening · 2026-07-02T10:32:34Z

+          tags: lfenergy/flexmeasures:${{ steps.version.outputs.tag }}
+
+      - name: Build and push (latest)
+        if: ${{ !github.event.release.prerelease }}


I don't know what a release.prerelease event is

It's not an event, it's a field — github.event.release.prerelease is a boolean GitHub sets on the release webhook payload, true/false depending on whether "Set as a pre-release" was checked when the GitHub Release was created. Added a comment explaining this inline.

nhoening · 2026-07-02T10:33:11Z

+#     flexmeasures-client's FLEXMEASURES_CLI_CMD/FLEXMEASURES_CLI_CONFIG_DIR
+#     support, used below to run that CLI inside the `server` container)
+#
+# This does not replace manual QA (e.g. UI login/graph checks, tutorial 5,


tutorial 5 can be covered, see other comment

Resolved via the same change — tutorial 5's runner script is now included, so the QA workflow runs tutorials 1-5.

- release_process.rst: fix "cut" -> "done" wording; point QA at the in-repo documentation/tut/scripts tutorial runners (including tutorial 5) instead of the tsc repo, since those scripts live here now. - docker-qa.yml: run tutorials 1-5 from documentation/tut/scripts directly, dropping the tsc checkout step and tsc-ref input. - list-merged-prs-since-tag.sh: only list merged PRs not yet referenced by a `PR #NNNN` link in an existing changelog file, instead of listing every merged PR unconditionally. - docker-publish.yml: build the image once and push it under both the version tag and (for stable releases) `latest`, instead of building twice; add a comment explaining what `github.event.release.prerelease` is. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

nhoening

Thanks!

* feat(ci): automate PyPI smoke test Signed-off-by: F.N. Claessen <claessen@seita.nl> * feat(ci): add Docker Hub release workflow Signed-off-by: F.N. Claessen <claessen@seita.nl> * feat(ci): add script to list PRs since the given (or latest) git tag Signed-off-by: F.N. Claessen <claessen@seita.nl> * docs(ci): add dev section on the release process Signed-off-by: F.N. Claessen <claessen@seita.nl> * refactor(ci): move changelog helper script into ci/ folder Keep it alongside the other CI scripts (ci/setup-postgres.sh etc.) rather than a standalone bin/ directory. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * feat(ci): add manually-triggered QA workflow for tutorials and HEMS Runs tutorials 1-4 and the HEMS walkthrough against the local docker compose stack, using the runner scripts kept in FlexMeasures/tsc and the HEMS example kept in FlexMeasures/flexmeasures-client (not duplicated here). Manual trigger only, to avoid slowing down every push while still cutting release QA toil. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * feat(ci): make qa-release.yml also validate HEMS report generation The HEMS example's report-generation step shells out to a local flexmeasures CLI, which doesn't work as-is against a dockerized server. Bind-mount examples/HEMS/configs into the server container and point FLEXMEASURES_CLI_CMD/FLEXMEASURES_CLI_CONFIG_DIR (added in FlexMeasures/flexmeasures-client#qa/hems-report-cli-override) at it, so report generation runs for real instead of being silently skipped. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * docs: add changelog entry for release automation workflows Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * feat(ci): require manual approval for Docker Hub publish Gate docker-publish.yml behind the dockerhub GitHub environment, mirroring the approval gate already used by pypi-publish.yml. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * refactor(ci): rename workflows for clarity, move dummy-price script into ci/ Rename build.yml -> docker-build.yml and release.yml -> pypi-publish.yml so filenames reflect what they actually build/publish, and qa-release.yml -> docker-qa.yml to distinguish it from the release-publishing pair. Move generate-dummy-price.sh into ci/, alongside the other CI helper scripts, and update references accordingly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix: address PR #2260 review comments - release_process.rst: fix "cut" -> "done" wording; point QA at the in-repo documentation/tut/scripts tutorial runners (including tutorial 5) instead of the tsc repo, since those scripts live here now. - docker-qa.yml: run tutorials 1-5 from documentation/tut/scripts directly, dropping the tsc checkout step and tsc-ref input. - list-merged-prs-since-tag.sh: only list merged PRs not yet referenced by a `PR #NNNN` link in an existing changelog file, instead of listing every merged PR unconditionally. - docker-publish.yml: build the image once and push it under both the version tag and (for stable releases) `latest`, instead of building twice; add a comment explaining what `github.event.release.prerelease` is. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Signed-off-by: F.N. Claessen <claessen@seita.nl> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> (cherry picked from commit 870c560) Signed-off-by: F.N. Claessen <claessen@seita.nl>

Flix6x and others added 6 commits July 1, 2026 22:31

feat(ci): automate PyPI smoke test

cb6cdcd

Signed-off-by: F.N. Claessen <claessen@seita.nl>

feat(ci): add Docker Hub release workflow

122f3d0

Signed-off-by: F.N. Claessen <claessen@seita.nl>

feat(ci): add script to list PRs since the given (or latest) git tag

0aa0e20

Signed-off-by: F.N. Claessen <claessen@seita.nl>

docs(ci): add dev section on the release process

e6b55d0

Signed-off-by: F.N. Claessen <claessen@seita.nl>

refactor(ci): move changelog helper script into ci/ folder

d6d1aa5

Keep it alongside the other CI scripts (ci/setup-postgres.sh etc.) rather than a standalone bin/ directory. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Flix6x self-assigned this Jul 1, 2026

Flix6x added documentation Improvements or additions to documentation Continuous Integration labels Jul 1, 2026

Flix6x changed the title ~~Release workflow~~ Release workflows Jul 1, 2026

Flix6x mentioned this pull request Jul 1, 2026

Make HEMS report generation dockerizable + fix false-negative on async uploads FlexMeasures/flexmeasures-client#216

Merged

3 tasks

docs: add changelog entry for release automation workflows

ce2f7c6

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Flix6x requested a review from nhoening July 2, 2026 05:33

Flix6x and others added 2 commits July 2, 2026 10:22

feat(ci): require manual approval for Docker Hub publish

864d029

Gate docker-publish.yml behind the dockerhub GitHub environment, mirroring the approval gate already used by pypi-publish.yml. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

nhoening requested changes Jul 2, 2026

View reviewed changes

Flix6x requested a review from nhoening July 2, 2026 20:46

nhoening approved these changes Jul 2, 2026

View reviewed changes

Flix6x merged commit 870c560 into main Jul 3, 2026
11 of 12 checks passed

Flix6x deleted the ci/release-workflow branch July 3, 2026 07:08

Flix6x added this to the 0.33.1 milestone Jul 3, 2026

	This page describes how FlexMeasures releases are cut, and which parts are automated versus manual.
	This page describes how FlexMeasures releases are done, and which parts are automated versus manual.

Uh oh!

Conversation

Flix6x commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Look & Feel

How to test

Further Improvements

Related Items

Uh oh!

nhoening left a comment

Choose a reason for hiding this comment

Uh oh!

nhoening Jul 2, 2026 • edited by Flix6x Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nhoening Jul 2, 2026 • edited by Flix6x Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nhoening left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Flix6x commented Jul 1, 2026 •

edited

Loading

nhoening Jul 2, 2026 •

edited by Flix6x

Loading

nhoening Jul 2, 2026 •

edited by Flix6x

Loading