Skip to content

feat: Split flex-context settings by commodity for dynamic capacity scheduling#2172

Open
Ahmad-Wahid wants to merge 55 commits into
feat/multi-feed-stockfrom
dev/split-flexcontext-by-commodity
Open

feat: Split flex-context settings by commodity for dynamic capacity scheduling#2172
Ahmad-Wahid wants to merge 55 commits into
feat/multi-feed-stockfrom
dev/split-flexcontext-by-commodity

Conversation

@Ahmad-Wahid

@Ahmad-Wahid Ahmad-Wahid commented May 14, 2026

Copy link
Copy Markdown
Contributor

Description

This PR updates the dynamic consumption capacity scheduling test to use split, commodity-specific flex-context settings for electricity and gas. It adds dynamic electricity and gas price data so the scheduler can prefer the electric heater when electricity is cheaper and the gas boiler when gas is cheaper, while ensuring that the dynamic electricity site consumption capacity is only applied to electricity devices and does not incorrectly constrain gas devices.

  • Added changelog item in documentation/changelog.rst

Look & Feel

flex_context = {
        "commodities": [
            {
                "commodity": "electricity",
                "consumption-price": {
                    "sensor": consumption_price.id,
                },
                "production-price": {
                    "sensor": consumption_price.id,
                },
                "site-power-capacity": "1900 kW",
                "site-consumption-capacity": {
                    "sensor": dynamic_consumption_capacity.id,
                },
                "site-production-capacity": "100 kW",
                "site-consumption-breach-price": "100000 EUR/kW",
                "site-production-breach-price": "100000 EUR/kW",
                "inflexible-device-sensors": [building_raw_power.id],
            },
            {
                "commodity": "gas",
                "consumption-price": {
                    "sensor": gas_price.id,
                },
                "production-price": {
                    "sensor": gas_price.id,
                },
                # No electricity dynamic capacity here.
                "site-consumption-capacity": "100000 kW",
            },
        ],
        "relax-constraints": True,
    }

...

How to test

pytest -k test_simulation_with_dynamic_consumption_capacity

...

Further Improvements

...

Related Items

...


Sign-off

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on code under GPL or other license that is incompatible with FlexMeasures

… scheduler

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
…capacity scheduling test

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
@Ahmad-Wahid Ahmad-Wahid requested a review from Flix6x May 14, 2026 12:17
@Ahmad-Wahid Ahmad-Wahid self-assigned this May 14, 2026

@Flix6x Flix6x left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick review. Also please triage the failing test suite.

Comment thread flexmeasures/data/models/planning/storage.py Outdated
Comment thread flexmeasures/data/schemas/scheduling/__init__.py Outdated
Comment thread flexmeasures/data/schemas/scheduling/__init__.py
Ahmad-Wahid and others added 23 commits May 26, 2026 17:22
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
…odity-flex-context

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
…'gas-price' for electricity as commodity

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
…split-flexcontext-by-commodity

# Conflicts:
#	flexmeasures/data/models/planning/storage.py
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
… dedicated issue

Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Flix6x and others added 25 commits June 8, 2026 17:34
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
* fix: keep ems-constraints and fix the test cases

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Update flexmeasures/data/models/planning/storage.py

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* fix: update the comment and raise value error if ems_constraints_group is not passed

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

---------

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…oving old flex-context fields into the electricity commodity context

Signed-off-by: F.N. Claessen <claessen@seita.nl>
…d field)

Signed-off-by: F.N. Claessen <claessen@seita.nl>
…-context

Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
… gas

Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…odel

Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…, respectively, because it is possible that the entire flex-config is described in the db instead of the trigger message

Signed-off-by: F.N. Claessen <claessen@seita.nl>
* feat: support passing flex-context as a list (one flex-context per commodity)

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: set default flex-context commodity to electricity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: preserve field order in case schema is made OpenAPI compatible

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: reduce documented nesting when defining a flex-context per commodity

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* dev: add todos

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* style: flake8

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* scheduling: complete schema refactoring per PR #2235

- Add SensorReferenceSchema import
- Override relax_constraints default to True in CommodityFlexContextSchema
- Remove duplicate breach price and relax fields from FlexContextSchema
- Remove duplicate set_default_breach_prices method from FlexContextSchema

Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* tests: add comprehensive tests for schema refactoring

- Test aggregate-consumption and aggregate-production fields
- Test SharedSchema fields accessible in FlexContextSchema
- Test CommodityFlexContextSchema relax_constraints defaults to True
- Test shared currency logic for flex-context listings
- Test breach prices in both schemas
- Add noqa comment for existing unused variable

Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>

* scheduling: add shared currency validation for commodity contexts

- Add validator to check prices share same currency across all commodity contexts
- Fix test data to use actual fixture sensor names
- All new tests now passing

Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>

* docs: add aggregate fields to documentation and update tests

- Add aggregate-consumption and aggregate-production to scheduling.rst
- Exclude COMMODITY_FLEX_CONTEXT and COMMODITY_FLEX_MODEL from doc test (already documented as "commodity")
- Exclude aggregate fields from UI test (not yet supported in UI)

Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>

* chore: upgrade openapi-specs.json

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: move _try_to_convert_price_units to SharedSchema

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: some fields cannot use source filters; now instead of just having validators in place to forbid that, we actually stop having those fields in the schemas of those fields

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: mypy

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* delete: remove redundant tests

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: minimize diff

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: default flex-model and flex-context to empty lists, because it is possible that the entire flex-config is described in the db instead of the trigger message (this cherry-pick was adapted, because the flex-context moved from being documented as a dict to a list)

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: prefer unique sensor IDs in examples

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: clarify cross-reference

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: UI support for picking a
sensor for aggregate-consumption sensor and/or aggregate-production

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: for some reason, for flex-context fields, the sensor-only property is defined in the html in 3 places

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: test coverage for UI support of aggregate-consumption and aggregate-production

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: add deprecation instructions for aggregate-power

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: move down the aggregate-power field documentation

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* use different compatible units.

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* update comment

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* update comment

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* update comment

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* update the comment

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* chore: update openapi-specs.json again

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: add multi feed stock tutorial

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* remove extra chart explanation

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* fix: update flexmeasures version in openapi-specs

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>

* feat: add multi commodity tutorial

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Apply my own suggestions from code review

Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>

* docs: changelog entry

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* test: add comprehensive coverage for aggregate-consumption/production flex-context fields
- Add test_asset_trigger_and_get_aggregate_schedule to verify aggregate sensors get
  populated (currently xfail as StorageScheduler doesn't yet populate them)
- Add test_asset_trigger_with_multi_commodity_flex_context to verify aggregate sensors
  work with multi-commodity flex-context format (list of commodity dicts)
- Document that aggregate sensor population is missing logic in StorageScheduler
- Both tests verify the infrastructure and schema support works correctly

* fix: compute per-commodity aggregate power flows for aggregate-consumption and aggregate-production sensors

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* Include per-commodity inflexible devices in aggregate schedules

_compute_commodity_aggregate_schedules only added the top-level inflexible
devices to the electricity commodity, so a commodity's own inflexible demand
(e.g. a heat load) was excluded from its aggregate-consumption schedule. The
aggregate then reflected only that commodity's flexible devices. Mirror the
device enumeration used in _prepare so each commodity context's
inflexible-device-sensors are appended to that commodity at the matching
ems_schedule indices.

* test: update aggregate sensor tests to verify implementation
- Remove xfail markers from both tests now that StorageScheduler populates aggregate sensors
- Update test docstrings to reflect actual functionality
- Tests now verify that aggregate-consumption and aggregate-production sensors receive data
- First test covers single aggregate pair with dual devices
- Second test covers aggregates with multiple devices scheduling together

* feat: test commodities referenced in only the flex-model or only the flex-context

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: test logs should explain why response was unexpected

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* style: black

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: AssetTriggerSchema accepts list-like flex-contexts

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: skip aggregate computation for unused commodities
When a multi-commodity flex_context includes a commodity (e.g., heat)
but no devices exist for that commodity, the sum() of an empty list
returns integer 0 instead of a pandas Series. This caused AttributeError
when calling .round() on the result later in compute().
Fixed by checking if any devices contribute to the commodity before
attempting to aggregate. If no devices exist for a commodity, skip
creating an aggregate schedule for it.

Signed-off-by: F.N. Claessen <claessen@seita.nl>

---------

Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+copilot@users.noreply.github.com>
Co-authored-by: Ahmad Wahid <59763365+Ahmad-Wahid@users.noreply.github.com>
Co-authored-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
@Flix6x Flix6x added this to the 1.0.0 milestone Jul 1, 2026
Flix6x added 2 commits July 2, 2026 11:54
…lking about the device state

Signed-off-by: F.N. Claessen <claessen@seita.nl>
…e; add cross-references between tutorials

Signed-off-by: F.N. Claessen <claessen@seita.nl>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants