SPEC-011 — Opt-in InferencePool saturation autoscaling (KEDA)
Type: composition · Status: draft · depends on SPEC-004 (EndpointPicker, opt-in)
Adds an opt-in fourth KEDA trigger to the inference-service composition that scales a vLLM model on the InferencePool saturation gauge emitted by the EndpointPicker (EPP, Gateway API Inference Extension v1.5.0), gated on spec.gateway.endpointPicker.enabled. Models without the EPP keep today's three vLLM-raw triggers unchanged.
The saturation gauge is emitted only by the EPP, and the EPP is opt-in / default-off / enabled on zero models today. So the feature is strictly opt-in and layers on SPEC-004 without forcing an EPP rollout (decision CL-1). Motivation: SPEC-001 records the KV-cache trigger as inert on L4 GPUs (running-ratio always fires first) — a single pool-level gauge is a cleaner leading signal where the EPP runs.
Spec artifacts: docs/specs/011-inferencepool-saturation-keda/ (spec.md / plan.md / clarifications.md).
Next: /clarify (3 open questions — exact v1.5.0 saturation metric name, default threshold, ADD-vs-replace long-term) → /validate → implement.
SPEC-011 — Opt-in InferencePool saturation autoscaling (KEDA)
Type: composition · Status: draft · depends on SPEC-004 (EndpointPicker, opt-in)
Adds an opt-in fourth KEDA trigger to the
inference-servicecomposition that scales a vLLM model on the InferencePool saturation gauge emitted by the EndpointPicker (EPP, Gateway API Inference Extension v1.5.0), gated onspec.gateway.endpointPicker.enabled. Models without the EPP keep today's three vLLM-raw triggers unchanged.The saturation gauge is emitted only by the EPP, and the EPP is opt-in / default-off / enabled on zero models today. So the feature is strictly opt-in and layers on SPEC-004 without forcing an EPP rollout (decision CL-1). Motivation: SPEC-001 records the KV-cache trigger as inert on L4 GPUs (running-ratio always fires first) — a single pool-level gauge is a cleaner leading signal where the EPP runs.
Spec artifacts:
docs/specs/011-inferencepool-saturation-keda/(spec.md/plan.md/clarifications.md).Next:
/clarify(3 open questions — exact v1.5.0 saturation metric name, default threshold, ADD-vs-replace long-term) →/validate→ implement.