Make HEMS report generation dockerizable + fix false-negative on async uploads#216
Merged
Conversation
The HEMS tutorial's report-generation step shells out to a bare `flexmeasures` on PATH, which only works if that CLI is configured against the same database as the server being scripted against. Add FLEXMEASURES_CLI_CMD and FLEXMEASURES_CLI_CONFIG_DIR env vars so callers can point report generation at a `flexmeasures` running elsewhere, e.g. inside a Docker Compose service (`docker compose exec -T server flexmeasures`), which unblocks running this step as part of automated release QA. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
_post_sensor_data_file() hardcoded `status != 200` as the failure condition, but the sensors/<id>/data/upload endpoint can legitimately return 202 Accepted when the server processes the upload as an async job. This made every file upload look like a failure whenever the server queues the work, which is what silently broke the HEMS tutorial's data-loading step (PART 2) before it ever reached forecasting/scheduling/reporting. Match check_for_status()'s existing convention elsewhere in this client: accept any 2xx, only raise on a genuine non-2xx status. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Coverage Report for CI Build 28550912284Coverage remained the same at 96.359%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsNo coverage regressions found. Coverage Stats
💛 - Coveralls |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Joint effort with FlexMeasures/flexmeasures#2260, which adds a manually-triggered QA workflow that runs the HEMS tutorial against a dockerized FlexMeasures stack as part of release testing. Getting that working surfaced two independent problems in this repo, both fixed here (and both part of this PR's diff):
CLI-command override for report generation.
examples/HEMS/reporters.pyshells out to a bareflexmeasuresonPATH, which only works if that CLI is installed locally and configured against the same database as the server being scripted against (per the existing three-terminaldocs/HEMS.rstinstructions). That breaks down when the server runs elsewhere, e.g. inside a Docker Compose service. AddedFLEXMEASURES_CLI_CMDandFLEXMEASURES_CLI_CONFIG_DIRenv vars so callers can point report generation at aflexmeasuresprocess running anywhere (e.g.docker compose exec -T server flexmeasures), with file paths translated accordingly. Documented indocs/HEMS.rst.Bugfix: file upload falsely treated
202 Acceptedas an error.FlexMeasuresClient._post_sensor_data_file()hardcodedresponse.status != 200as its failure condition, butsensors/<id>/data/uploadcan legitimately return202 Acceptedwhen the server queues the upload as an async job. This silently broke the HEMS tutorial's very first data-loading step, so it never got anywhere near forecasting, scheduling, or reporting. Fixed to accept any 2xx status, matching the conventioncheck_for_status()already uses elsewhere in this client. Added a regression test (test_post_sensor_data_with_file_accepted).These two PRs should land together: FlexMeasures/flexmeasures#2260's QA workflow won't successfully complete its HEMS step until this PR is merged.
Test plan
uv run poe test— 183 passedblack --check/flake8clean on changed filesHEMS_setup.py) locally against a dockerized FlexMeasures server, usingFLEXMEASURES_CLI_CMD/FLEXMEASURES_CLI_CONFIG_DIRpointed atdocker compose exec -T server flexmeasures: all 6 simulation steps completed, all data uploads succeeded (no more false "202" failures), all report-generation calls (AggregatorReporter,PandasReporterx2 per step) succeeded via the dockerized CLI, ending in "HEMS Tutorial completed successfully!" with zero reporter/upload failures.🤖 Generated with Claude Code