feat: Split flex-context settings by commodity for dynamic capacity scheduling#2172
Open
Ahmad-Wahid wants to merge 55 commits into
Open
feat: Split flex-context settings by commodity for dynamic capacity scheduling#2172Ahmad-Wahid wants to merge 55 commits into
Ahmad-Wahid wants to merge 55 commits into
Conversation
… scheduler Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
…capacity scheduling test Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Flix6x
requested changes
May 14, 2026
Flix6x
left a comment
Member
There was a problem hiding this comment.
Quick review. Also please triage the failing test suite.
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
…odity-flex-context Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
…'gas-price' for electricity as commodity Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
…split-flexcontext-by-commodity # Conflicts: # flexmeasures/data/models/planning/storage.py
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
…split-flexcontext-by-commodity
…split-flexcontext-by-commodity
…split-flexcontext-by-commodity
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
… dedicated issue Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
* fix: keep ems-constraints and fix the test cases Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com> * Update flexmeasures/data/models/planning/storage.py Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com> Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com> * fix: update the comment and raise value error if ems_constraints_group is not passed Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com> --------- Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com> Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com> Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com> Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…oving old flex-context fields into the electricity commodity context Signed-off-by: F.N. Claessen <claessen@seita.nl>
…d field) Signed-off-by: F.N. Claessen <claessen@seita.nl>
…-context Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
… gas Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…odel Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…, respectively, because it is possible that the entire flex-config is described in the db instead of the trigger message Signed-off-by: F.N. Claessen <claessen@seita.nl>
3 tasks
* feat: support passing flex-context as a list (one flex-context per commodity) Signed-off-by: F.N. Claessen <claessen@seita.nl> * fix: set default flex-context commodity to electricity Signed-off-by: F.N. Claessen <claessen@seita.nl> * fix: preserve field order in case schema is made OpenAPI compatible Signed-off-by: F.N. Claessen <claessen@seita.nl> * feat: reduce documented nesting when defining a flex-context per commodity Signed-off-by: F.N. Claessen <claessen@seita.nl> * dev: add todos Signed-off-by: F.N. Claessen <claessen@seita.nl> * style: flake8 Signed-off-by: F.N. Claessen <claessen@seita.nl> * scheduling: complete schema refactoring per PR #2235 - Add SensorReferenceSchema import - Override relax_constraints default to True in CommodityFlexContextSchema - Remove duplicate breach price and relax fields from FlexContextSchema - Remove duplicate set_default_breach_prices method from FlexContextSchema Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com> Signed-off-by: F.N. Claessen <claessen@seita.nl> * tests: add comprehensive tests for schema refactoring - Test aggregate-consumption and aggregate-production fields - Test SharedSchema fields accessible in FlexContextSchema - Test CommodityFlexContextSchema relax_constraints defaults to True - Test shared currency logic for flex-context listings - Test breach prices in both schemas - Add noqa comment for existing unused variable Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com> * scheduling: add shared currency validation for commodity contexts - Add validator to check prices share same currency across all commodity contexts - Fix test data to use actual fixture sensor names - All new tests now passing Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com> * docs: add aggregate fields to documentation and update tests - Add aggregate-consumption and aggregate-production to scheduling.rst - Exclude COMMODITY_FLEX_CONTEXT and COMMODITY_FLEX_MODEL from doc test (already documented as "commodity") - Exclude aggregate fields from UI test (not yet supported in UI) Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com> * chore: upgrade openapi-specs.json Signed-off-by: F.N. Claessen <claessen@seita.nl> * fix: move _try_to_convert_price_units to SharedSchema Signed-off-by: F.N. Claessen <claessen@seita.nl> * fix: some fields cannot use source filters; now instead of just having validators in place to forbid that, we actually stop having those fields in the schemas of those fields Signed-off-by: F.N. Claessen <claessen@seita.nl> * chore: mypy Signed-off-by: F.N. Claessen <claessen@seita.nl> * delete: remove redundant tests Signed-off-by: F.N. Claessen <claessen@seita.nl> * fix: minimize diff Signed-off-by: F.N. Claessen <claessen@seita.nl> * fix: default flex-model and flex-context to empty lists, because it is possible that the entire flex-config is described in the db instead of the trigger message (this cherry-pick was adapted, because the flex-context moved from being documented as a dict to a list) Signed-off-by: F.N. Claessen <claessen@seita.nl> * docs: prefer unique sensor IDs in examples Signed-off-by: F.N. Claessen <claessen@seita.nl> * docs: clarify cross-reference Signed-off-by: F.N. Claessen <claessen@seita.nl> * feat: UI support for picking a sensor for aggregate-consumption sensor and/or aggregate-production Signed-off-by: F.N. Claessen <claessen@seita.nl> * fix: for some reason, for flex-context fields, the sensor-only property is defined in the html in 3 places Signed-off-by: F.N. Claessen <claessen@seita.nl> * feat: test coverage for UI support of aggregate-consumption and aggregate-production Signed-off-by: F.N. Claessen <claessen@seita.nl> * docs: add deprecation instructions for aggregate-power Signed-off-by: F.N. Claessen <claessen@seita.nl> * docs: move down the aggregate-power field documentation Signed-off-by: F.N. Claessen <claessen@seita.nl> * use different compatible units. Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com> Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com> * update comment Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com> Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com> * update comment Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com> Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com> * update comment Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com> Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com> * update the comment Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com> Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com> * chore: update openapi-specs.json again Signed-off-by: F.N. Claessen <claessen@seita.nl> * feat: add multi feed stock tutorial Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com> * remove extra chart explanation Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com> * fix: update flexmeasures version in openapi-specs Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com> Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com> * feat: add multi commodity tutorial Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com> * Apply my own suggestions from code review Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com> Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com> * docs: changelog entry Signed-off-by: F.N. Claessen <claessen@seita.nl> * test: add comprehensive coverage for aggregate-consumption/production flex-context fields - Add test_asset_trigger_and_get_aggregate_schedule to verify aggregate sensors get populated (currently xfail as StorageScheduler doesn't yet populate them) - Add test_asset_trigger_with_multi_commodity_flex_context to verify aggregate sensors work with multi-commodity flex-context format (list of commodity dicts) - Document that aggregate sensor population is missing logic in StorageScheduler - Both tests verify the infrastructure and schema support works correctly * fix: compute per-commodity aggregate power flows for aggregate-consumption and aggregate-production sensors Signed-off-by: F.N. Claessen <claessen@seita.nl> * Include per-commodity inflexible devices in aggregate schedules _compute_commodity_aggregate_schedules only added the top-level inflexible devices to the electricity commodity, so a commodity's own inflexible demand (e.g. a heat load) was excluded from its aggregate-consumption schedule. The aggregate then reflected only that commodity's flexible devices. Mirror the device enumeration used in _prepare so each commodity context's inflexible-device-sensors are appended to that commodity at the matching ems_schedule indices. * test: update aggregate sensor tests to verify implementation - Remove xfail markers from both tests now that StorageScheduler populates aggregate sensors - Update test docstrings to reflect actual functionality - Tests now verify that aggregate-consumption and aggregate-production sensors receive data - First test covers single aggregate pair with dual devices - Second test covers aggregates with multiple devices scheduling together * feat: test commodities referenced in only the flex-model or only the flex-context Signed-off-by: F.N. Claessen <claessen@seita.nl> * feat: test logs should explain why response was unexpected Signed-off-by: F.N. Claessen <claessen@seita.nl> * style: black Signed-off-by: F.N. Claessen <claessen@seita.nl> * feat: AssetTriggerSchema accepts list-like flex-contexts Signed-off-by: F.N. Claessen <claessen@seita.nl> * fix: skip aggregate computation for unused commodities When a multi-commodity flex_context includes a commodity (e.g., heat) but no devices exist for that commodity, the sum() of an empty list returns integer 0 instead of a pandas Series. This caused AttributeError when calling .round() on the result later in compute(). Fixed by checking if any devices contribute to the commodity before attempting to aggregate. If no devices exist for a commodity, skip creating an aggregate schedule for it. Signed-off-by: F.N. Claessen <claessen@seita.nl> --------- Signed-off-by: F.N. Claessen <claessen@seita.nl> Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com> Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com> Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+copilot@users.noreply.github.com> Co-authored-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com> Co-authored-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
…lking about the device state Signed-off-by: F.N. Claessen <claessen@seita.nl>
…e; add cross-references between tutorials Signed-off-by: F.N. Claessen <claessen@seita.nl>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR updates the dynamic consumption capacity scheduling test to use split, commodity-specific flex-context settings for electricity and gas. It adds dynamic electricity and gas price data so the scheduler can prefer the electric heater when electricity is cheaper and the gas boiler when gas is cheaper, while ensuring that the dynamic electricity site consumption capacity is only applied to electricity devices and does not incorrectly constrain gas devices.
documentation/changelog.rstLook & Feel
...
How to test
...
Further Improvements
...
Related Items
...
Sign-off