Feat: dynamic soc bounds defaults

documentation/api/introduction.rst

@@ -124,7 +124,8 @@ The response body will contain a JSON message with a ``status`` field set to ``"
 .. note::
 
     FlexMeasures' built-in storage scheduler no longer computes a fallback schedule for infeasible problems.
-    Instead, ``soc-minima`` and ``soc-maxima`` are relaxed by default through ``"relax-soc-constraints": true``, while ``soc-min``, ``soc-max`` and ``soc-targets`` remain hard constraints.
+    Instead, dynamic ``soc-min`` and ``soc-max`` boundaries, including the legacy aliases ``soc-minima`` and ``soc-maxima``, are relaxed by default through ``"relax-soc-constraints": true``.
+    Fixed ``soc-min`` / ``soc-max`` values and exact ``soc-targets`` remain hard constraints.
     If hard constraints cannot be satisfied, the scheduling job fails and clients receive the failure reason when requesting the schedule.
 
     For custom schedulers that still define a fallback scheduler, server administrators can configure whether clients receive a 303 redirect (``FLEXMEASURES_FALLBACK_REDIRECT = True``) or whether FlexMeasures follows the fallback automatically and returns the fallback schedule directly (``FLEXMEASURES_FALLBACK_REDIRECT = False``, the default).

documentation/changelog.rst

@@ -18,6 +18,7 @@ New features
 * Let storage scheduling infer missing ``power-capacity`` from directional device capacities before falling back to site capacity, and default the missing opposite capacity to zero when only a non-zero ``consumption-capacity`` or ``production-capacity`` is configured [see `PR #2222 <https://www.github.com/FlexMeasures/flexmeasures/pull/2222>`_]
 * CLI support for adding/editing account attributes [see `PR #2242 <https://www.github.com/FlexMeasures/flexmeasures/pull/2242>`_]
 * Relax storage SoC constraints by default and report infeasible storage schedules directly instead of saving fallback schedules [see `PR #2252 <https://www.github.com/FlexMeasures/flexmeasures/pull/2252>`_]
+* Support defining dynamic storage ``soc-min`` and ``soc-max`` boundaries with sensor references or time series, including sensor ``default`` fallbacks; ``soc-minima`` and ``soc-maxima`` remain supported as legacy aliases [see `PR #2267 <https://www.github.com/FlexMeasures/flexmeasures/pull/2267>`_]
 
 Infrastructure / Support
 ----------------------

documentation/features/scheduling.rst

@@ -264,13 +264,14 @@ If you model devices that *buffer* energy (e.g. thermal energy storage systems c
 However, here are some tips to model a buffer correctly:
 
    - Describe the thermal energy content in kWh or MWh.
-   - Set ``soc-minima`` to the accumulative usage forecast.
+   - Set dynamic ``soc-min`` values to the accumulative usage forecast.
    - Set ``charging-efficiency`` to the sensor describing the :abbr:`COP (coefficient of performance)` values.
    - Set ``storage-efficiency`` to a value below 100% to model (heat) loss.
 
 If the flex model describes an infeasible problem for the storage scheduler, the failure should remain visible.
-By default, ``soc-minima`` and ``soc-maxima`` are relaxed into soft constraints, so the scheduler can still return a useful schedule when these boundaries cannot be fully met.
-Exact ``soc-targets`` and physical ``soc-min`` / ``soc-max`` bounds remain hard constraints.
+By default, dynamic ``soc-min`` and ``soc-max`` boundaries are relaxed into soft constraints, so the scheduler can still return a useful schedule when these boundaries cannot be fully met.
+The legacy ``soc-minima`` and ``soc-maxima`` aliases follow the same behavior.
+Exact ``soc-targets`` and fixed physical ``soc-min`` / ``soc-max`` bounds remain hard constraints.
 If those hard constraints make the problem infeasible, the scheduling job fails instead of producing a fallback schedule.
 
 It is important to take note of these failures. Often, misconfigured flex models are the reason.

flexmeasures/api/v3_0/assets.py

@@ -1433,7 +1433,7 @@ def trigger_schedule(
                               power-capacity: 25 kW
                               consumption-capacity: {sensor: 42}
                               production-capacity: 30 kW
-                              soc-minima:
+                              soc-min:
                                 - {start: "2015-06-02T12:00:00+00:00", end: "2015-06-02T13:00:00+00:00", value: 10 kWh}
                             - sensor: 932
                               consumption-capacity: 0 kW

flexmeasures/api/v3_0/sensors.py

@@ -940,9 +940,9 @@ def trigger_schedule(
                         soc-targets:
                           - value: "25 kWh"
                             datetime: "2015-06-02T16:00:00+00:00"
-                        soc-minima:
+                        soc-min:
                           sensor: 300
-                        soc-min: "10 kWh"
+                          default: "10 kWh"
                         soc-max: "25 kWh"
                         charging-efficiency: "120%"
                         discharging-efficiency:

flexmeasures/data/models/planning/storage.py

@@ -1598,9 +1598,9 @@ def _build_soc_schedule(
         For sensors with a '%' unit, the soc-max flex-model field is used as capacity.
         If soc-max is missing or zero for a '%' sensor, the schedule is skipped with a warning.
 
-        Note: soc-max is a QuantityField (not a VariableQuantityField), so it is always a float
-        after deserialization and cannot be a sensor reference. The isinstance guard below is
-        therefore a defensive check for forward-compatibility.
+        Note: dynamic soc-max values are routed to soc_maxima during deserialization, so
+        soc_max is still a fixed float after deserialization. The isinstance guard below
+        is therefore a defensive check for malformed data.
         """
         soc_schedule = {}
         for d, flex_model_d in enumerate(flex_model):

flexmeasures/data/models/planning/tests/test_solver.py

@@ -2452,6 +2452,17 @@ def compute_schedule(flex_model):
     # because soc-maxima = soc-minima = soc-targets
     assert all(abs(soc[8:].values - expected_soc_schedule) < 1e-5)
 
+    # remove legacy soc-minima/soc-maxima and use dynamic canonical soc-min/soc-max
+    del flex_model["soc-minima"]
+    del flex_model["soc-maxima"]
+    flex_model["soc-min"] = {"sensor": soc_minima.id, "default": "0 MWh"}
+    flex_model["soc-max"] = {"sensor": soc_maxima.id, "default": "10 MWh"}
+    schedule = compute_schedule(flex_model)
+
+    soc = check_constraints(power, schedule, soc_at_start)
+
+    assert all(abs(soc[8:].values - expected_soc_schedule) < 1e-5)
+
 
 @pytest.mark.parametrize("unit", [None, "MWh", "kWh"])
 @pytest.mark.parametrize("soc_unit", ["kWh", "MWh"])
@@ -2537,7 +2548,7 @@ def test_battery_storage_different_units(
 
 
 @pytest.mark.parametrize(
-    "ts_field, ts_specs",
+    "ts_field, ts_specs, expected_charge, expected_discharge",
     [
         # The battery only has time to charge up to 950 kWh halfway
         (
@@ -2549,6 +2560,8 @@ def test_battery_storage_different_units(
                     "value": "850 kW",
                 }
             ],
+            0.85,
+            -0.85,
         ),
         # Same, but the event time is specified with a duration instead of an end time
         (
@@ -2560,6 +2573,8 @@ def test_battery_storage_different_units(
                     "value": "850 kW",
                 }
             ],
+            0.85,
+            -0.85,
         ),
         # Can only charge up to 950 kWh halfway
         (
@@ -2570,6 +2585,20 @@ def test_battery_storage_different_units(
                     "value": "950 kWh",
                 }
             ],
+            0.85,
+            -0.85,
+        ),
+        # Same dynamic maximum through the canonical soc-max field
+        (
+            "soc-max",
+            [
+                {
+                    "datetime": "2015-01-02T16:00+01",
+                    "value": "950 kWh",
+                }
+            ],
+            0.85,
+            -0.85,
         ),
         # Must end up at a maximum of 200 kWh, for which it is cheapest to charge to 950 and then to discharge to 200
         (
@@ -2581,6 +2610,47 @@ def test_battery_storage_different_units(
                     "value": "200 kWh",
                 }
             ],
+            0.85,
+            -0.85,
+        ),
+        # Same dynamic maximum through the canonical soc-max field
+        (
+            "soc-max",
+            [
+                {
+                    "start": "2015-01-02T16:45+01",
+                    "duration": "PT15M",
+                    "value": "200 kWh",
+                }
+            ],
+            0.85,
+            -0.85,
+        ),
+        # Must end up at a minimum of 200 kWh, so it is cheapest to fill completely and then discharge to 200
+        (
+            "soc-minima",
+            [
+                {
+                    "start": "2015-01-02T16:45+01",
+                    "duration": "PT15M",
+                    "value": "200 kWh",
+                }
+            ],
+            0.9,
+            -0.8,
+        ),
+        # Same dynamic minimum through the canonical soc-min field
+        (
+            "soc-min",
+            [
+                {
+                    "start": "2015-01-02T16:45+01",
+                    "duration": "PT15M",
+                    "value": "200 kWh",
+                }
+            ],
+            0.9,
+            -0.8,
         ),
     ],
 )
@@ -2589,6 +2659,8 @@ def test_battery_storage_with_time_series_in_flex_model(
     db,
     ts_field,
     ts_specs,
+    expected_charge,
+    expected_discharge,
 ):
     """
     Test scheduling a 1 MWh battery for 2h with a low -> high price transition with
@@ -2638,14 +2710,8 @@ def test_battery_storage_with_time_series_in_flex_model(
     soc_at_start = ur.Quantity(soc_at_start).to("MWh").magnitude
     check_constraints(battery, schedule, soc_at_start)
 
-    # charge 850 kWh in the cheap price period (100 kWh -> 950kWh)
-    assert schedule[:4].sum() * 0.25 == pytest.approx(0.85)
-
-    # discharge fully or to what's needed in the expensive price period (950 kWh -> 100 or 200 kWh)
-    if ts_field == "soc-minima":
-        assert schedule[4:].sum() * 0.25 == pytest.approx(-0.75)
-    else:
-        assert schedule[4:].sum() * 0.25 == pytest.approx(-0.85)
+    assert schedule[:4].sum() * 0.25 == pytest.approx(expected_charge)
+    assert schedule[4:].sum() * 0.25 == pytest.approx(expected_discharge)
 
 
 def test_unavoidable_capacity_breach():

flexmeasures/data/models/planning/tests/test_utils_fresh_db.py

@@ -8,6 +8,7 @@
 from flexmeasures.data.models.data_sources import DataSource
 from flexmeasures.data.schemas.sensors import SensorReference
 from flexmeasures.data.models.planning.utils import get_series_from_quantity_or_sensor
+from flexmeasures.utils.unit_utils import ur
 
 
 def test_get_series_from_sensor_reference_source_filter_integration(fresh_db):
@@ -86,6 +87,48 @@ def test_get_series_from_sensor_reference_source_filter_integration(fresh_db):
     assert result_forecaster.iloc[0] == pytest.approx(200.0)
 
 
+def test_get_series_from_sensor_reference_default_fills_missing_values(fresh_db):
+    """A SensorReference default fills query slots with no matching sensor belief."""
+    query_window = (
+        pd.Timestamp("2025-06-01 08:00:00+02:00"),
+        pd.Timestamp("2025-06-01 08:30:00+02:00"),
+    )
+    source = DataSource(name="test-default-source", type="scheduler")
+    fresh_db.session.add(source)
+    asset_type = GenericAssetType(name="test-asset-type-default")
+    fresh_db.session.add(asset_type)
+    asset = GenericAsset(name="test-asset-default", generic_asset_type=asset_type)
+    fresh_db.session.add(asset)
+    sensor = Sensor(
+        name="test-sensor-default",
+        generic_asset=asset,
+        event_resolution=timedelta(minutes=15),
+        unit="MW",
+    )
+    fresh_db.session.add(sensor)
+    fresh_db.session.flush()
+    fresh_db.session.add(
+        TimedBelief(
+            event_start=query_window[0],
+            belief_horizon=timedelta(0),
+            event_value=0.1,
+            source=source,
+            sensor=sensor,
+        )
+    )
+    fresh_db.session.commit()
+
+    result = get_series_from_quantity_or_sensor(
+        variable_quantity=SensorReference(sensor=sensor, default=ur.Quantity("1 MW")),
+        query_window=query_window,
+        resolution=sensor.event_resolution,
+        unit="kW",
+        as_instantaneous_events=False,
+    )
+
+    assert list(result) == pytest.approx([100.0, 1000.0])
+
+
 def test_get_series_from_sensor_reference_sources_filter_integration(fresh_db):
     """A :class:`SensorReference` with ``sources`` returns only beliefs from the specified source.
 

flexmeasures/data/models/planning/utils.py

@@ -299,6 +299,14 @@ def get_series_from_quantity_or_sensor(
         time_series = convert_units(
             time_series, variable_quantity.unit, unit, resolution
         )
+        if variable_quantity.default is not None:
+            default_value = convert_units(
+                variable_quantity.default.magnitude,
+                str(variable_quantity.default.units),
+                unit,
+                resolution,
+            )
+            time_series = time_series.fillna(default_value)
     elif isinstance(variable_quantity, Sensor):
         bdf: tb.BeliefsDataFrame = TimedBelief.search(
             variable_quantity,

flexmeasures/data/schemas/scheduling/__init__.py

@@ -611,7 +611,7 @@ def _to_currency_per_mwh(price_unit: str) -> str:
         "description": rst_to_openapi(metadata.SOC_MIN.description),
         "types": {
             "backend": "typeThree",
-            "ui": "One fixed value or a dynamic signal (via a sensor).",
+            "ui": "A fixed lower boundary or a dynamic lower boundary with an optional default fallback.",
         },
         "example-units": EXAMPLE_UNIT_TYPES["energy"],
     },
@@ -620,7 +620,7 @@ def _to_currency_per_mwh(price_unit: str) -> str:
         "description": rst_to_openapi(metadata.SOC_MAX.description),
         "types": {
             "backend": "typeThree",
-            "ui": "One fixed value or a dynamic signal (via a sensor).",
+            "ui": "A fixed upper boundary or a dynamic upper boundary with an optional default fallback.",
         },
         "example-units": EXAMPLE_UNIT_TYPES["energy"],
     },
@@ -629,7 +629,7 @@ def _to_currency_per_mwh(price_unit: str) -> str:
         "description": rst_to_openapi(metadata.SOC_MINIMA.description),
         "types": {
             "backend": "typeTwo",
-            "ui": "A sensor which records the state of charge.",
+            "ui": "Deprecated alias for dynamic soc-min values.",
         },
         "example-units": EXAMPLE_UNIT_TYPES["energy"],
     },
@@ -638,7 +638,7 @@ def _to_currency_per_mwh(price_unit: str) -> str:
         "description": rst_to_openapi(metadata.SOC_MAXIMA.description),
         "types": {
             "backend": "typeTwo",
-            "ui": "A sensor which records the state of charge.",
+            "ui": "Deprecated alias for dynamic soc-max values.",
         },
         "example-units": EXAMPLE_UNIT_TYPES["energy"],
     },