Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 10 additions & 11 deletions docs-site/src/content/docs/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,20 +254,18 @@ instead of growing memory without a bound.

## Default reasoning effort

`defaultEffort` supplies `reasoning.effort` only when all of these are true:
`defaultEffort` supplies `reasoning.effort` when the combo has a non-null default and the selected target advertises a compatible effort. By default, `defaultEffortMode: "fallback"` preserves an explicit caller effort. Set `defaultEffortMode: "force"` to make the operator-configured default override a valid caller effort (for example, caller `medium` becomes configured `max`). Force mode can increase cost and latency, is available only through combo configuration/management, and is rejected without `defaultEffort`.

1. the combo has a non-null default;
2. the caller did not set an effort; and
3. the selected target's catalog advertises that exact effort.
Resolution remains capability-safe: the configured effort is lowered to the target's highest compatible rung, an explicitly unsupported target receives no effort control, and an unknown capability never causes an override or injection. Malformed caller effort is not repaired into a valid expensive request.

If the request has no `reasoning` object, opencodex creates one. If `reasoning` exists without an
`effort` property, it preserves the other fields and adds the default. A caller-provided effort is
never overwritten.
`effort` property, it preserves the other fields and adds the default. A valid caller-provided effort
is overwritten only in explicit `force` mode.

When target capability is unknown or does not include the configured effort, opencodex omits the
default and leaves the target's own behavior unchanged. Supported values are `low`, `medium`,
`high`, `xhigh`, `max`, and `ultra`; omit the field or set it to `null` to leave effort entirely to
the caller and target.
When target capability is unknown, opencodex leaves the request unchanged. When the target
explicitly advertises no effort control, opencodex omits the effort. Supported values are `low`,
`medium`, `high`, `xhigh`, `max`, and `ultra`; omit `defaultEffort` or set it to `null` to leave effort
entirely to the caller and target.

### Mixed-capability groups (`reasoningEffortMode`)

Expand Down Expand Up @@ -414,7 +412,8 @@ Combos are stored in the top-level `combos` object, keyed by combo id:
| `stickyLimit` | No | `1` | Integer from 1 to 100 successful requests per round-robin selection. Applies only to round-robin. |
| `cooldownMs` | No | unset → upstream fallback (5 s for request-rate 429 codes `1302`/`1305`, otherwise 60 s) | Integer from 1 to 600000. When set, applies as the per-target cooldown whenever no usable upstream `Retry-After` or Codex reset signal exists, including request-rate 429s; when unset, uses the upstream fallback. |
| `waitForCooldownMs` | No | `0` | Integer from 0 to 600000. Maximum time to wait for the earliest eligible cooling target before returning `combo_unavailable`; abort cancels the wait. |
| `defaultEffort` | No | `null` | `low`, `medium`, `high`, `xhigh`, `max`, or `ultra`; applied only when the caller omits effort and the target advertises support. |
| `defaultEffort` | No | `null` | `low`, `medium`, `high`, `xhigh`, `max`, or `ultra`; resolved against each target's advertised ladder. |
| `defaultEffortMode` | No | `"fallback"` | `"fallback"` preserves an explicit caller effort. `"force"` overrides valid caller effort with `defaultEffort` and requires a non-null default; it can increase cost and latency. |
| `reasoningEffortMode` | No | `"strict"` | `"strict"` intersects every known target ladder, so one target advertising no effort control empties the combo's picker. `"adaptive"` excludes those empty ladders from the published intersection. Metadata only; dispatch is unchanged. |
| `imageInput` | No | `"auto"` | `"auto"` or `"disabled"`. `"auto"` publishes image support only when every target supports images; `"disabled"` forces text-only (drops image from published modalities and rejects image-bearing requests before dispatch). |
| `alias` | No | none | Optional trimmed public model id; use the alias rules above. An empty value is stored as no alias. |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,8 @@ namespace, and cannot use reserved bare native families such as `gpt-*`, `o1-*`,
| `stickyLimit?` | `number` | `1` | Successful requests retained in one round-robin batch. Range 1–100. Applies only to round-robin. |
| `cooldownMs?` | `number` | unset → upstream fallback (5 s for request-rate 429 codes `1302`/`1305`, otherwise 60 s) | Range 1–600000. When set, applies whenever no usable upstream `Retry-After` or Codex reset signal exists, including request-rate 429s; when unset, uses the upstream fallback. Upstream signals take precedence and all cooldowns are capped at 10 minutes. |
| `waitForCooldownMs?` | `number` | `0` | Maximum wait for the earliest eligible cooling target on each selection attempt before returning `combo_unavailable`. Range 0–600000; an abort cancels the wait. |
| `defaultEffort?` | `"low" \| "medium" \| "high" \| "xhigh" \| "max" \| "ultra" \| null` | unset | Applied only when the caller omits effort and the selected target advertises the requested rung. |
| `defaultEffort?` | `"low" \| "medium" \| "high" \| "xhigh" \| "max" \| "ultra" \| null` | unset | Resolved against each selected target's advertised effort ladder. |
| `defaultEffortMode?` | `"fallback" \| "force"` | `"fallback"` | `"fallback"` preserves explicit caller effort. `"force"` overrides valid caller effort with `defaultEffort`, requires a non-null default, and can increase cost and latency. Unknown target capability fails closed. |
| `reasoningEffortMode?` | `"strict" \| "adaptive"` | `"strict"` | `"strict"` intersects every known target effort ladder, so a target advertising no effort control empties the combo's picker. `"adaptive"` excludes those empty ladders from the published intersection. Picker metadata only; target selection and dispatch are unchanged. |
| `imageInput?` | `"auto" \| "disabled"` | `"auto"` | `"auto"` publishes image only when every target supports images; `"disabled"` forces text-only (drops image from published modalities and rejects image-bearing requests before dispatch). |
| `alias?` | `string` | — | Optional public model id in place of the canonical picker slug. |
Expand Down
11 changes: 10 additions & 1 deletion src/cli/combo.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ const USAGE = `Usage:
ocx combo show <id> [--json]
ocx combo set <id> --targets <provider/model[:weight],...>
[--strategy <failover|round-robin|random|least-used|reset-window>] [--sticky <1-100>]
[--effort <low|medium|high|xhigh|max|ultra|->] [--alias <name|->]
[--effort <low|medium|high|xhigh|max|ultra|->] [--effort-mode <fallback|force>]
(force overrides valid client effort and can increase cost/latency) [--alias <name|->]
[--native-alias] [--display-name <label|->]
[--rename-from <id>] [--json]
ocx combo remove <id> --yes [--json]`;
Expand Down Expand Up @@ -80,6 +81,10 @@ async function set(argv: string[], deps: RuntimeApiDeps): Promise<void> {
if (strategy !== "round-robin") throw new CliUsageError("--sticky applies only to round-robin", USAGE);
}
const effort = takeOption(args, "--effort");
const effortMode = takeOption(args, "--effort-mode");
if (effortMode !== undefined && effortMode !== "fallback" && effortMode !== "force") {
throw new CliUsageError("--effort-mode must be fallback or force", USAGE);
}
const alias = takeOption(args, "--alias");
const nativeAlias = takeFlag(args, "--native-alias");
const displayName = takeOption(args, "--display-name");
Expand All @@ -91,12 +96,16 @@ async function set(argv: string[], deps: RuntimeApiDeps): Promise<void> {
targets: parseTargets(targetsRaw),
};
if (effort !== undefined) combo.defaultEffort = effort === "-" ? null : effort;
if (effortMode !== undefined) combo.defaultEffortMode = effortMode;
if (alias !== undefined) combo.alias = alias === "-" ? "" : alias;
if (nativeAlias) combo.nativeAlias = true;
if (displayName !== undefined) combo.displayName = displayName === "-" ? "" : displayName;
const current = await runtimeRequest<{ combos?: ComboRow[] }>("/api/combos", {}, deps);
const existing = (current.combos ?? []).find(row => row.id === (renameFrom ?? id));
if (existing?.imageInput === "disabled") combo.imageInput = "disabled";
if (effortMode === undefined && existing?.defaultEffortMode === "force") {
combo.defaultEffortMode = effort === "-" ? "fallback" : "force";
}
const result = await runtimeRequest("/api/combos", {
method: "PUT",
body: JSON.stringify({ id, combo, ...(renameFrom ? { renameFrom } : {}) }),
Expand Down
27 changes: 17 additions & 10 deletions src/combos/request.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import type { OcxComboDefaultEffort, OcxComboTarget, OcxConfig } from "../types";
import { resolveEffortAtOrBelow } from "../reasoning-effort";
import type { OcxComboDefaultEffort, OcxComboDefaultEffortMode, OcxComboTarget, OcxConfig } from "../types";
import { isCodexReasoningEffort, resolveEffortAtOrBelow } from "../reasoning-effort";
import { resolveComboId } from "./types";

const warnedUnsupportedDefaults = new Set<string>();
Expand Down Expand Up @@ -59,18 +59,25 @@ export function concreteComboRequestBody(
target: Pick<OcxComboTarget, "provider" | "model">,
defaultEffort: OcxComboDefaultEffort | null,
targetReasoningEfforts: readonly string[] | undefined,
defaultEffortMode: OcxComboDefaultEffortMode = "fallback",
): Record<string, unknown> {
const clone = structuredClone(body) as Record<string, unknown>;
clone.model = `${target.provider}/${target.model}`;
if (!defaultEffort) return clone;
if (defaultEffortMode === "force" && (!defaultEffort || !isCodexReasoningEffort(defaultEffort))) {
throw new Error("force combo default effort requires a valid defaultEffort");
}
if (!defaultEffort || !isCodexReasoningEffort(defaultEffort)) return clone;
const reasoning = clone.reasoning;
const needsDefault = reasoning === undefined || (
reasoning
&& typeof reasoning === "object"
&& !Array.isArray(reasoning)
&& !Object.prototype.hasOwnProperty.call(reasoning, "effort")
);
if (!needsDefault) return clone;
const reasoningRecord = reasoning && typeof reasoning === "object" && !Array.isArray(reasoning)
? reasoning as Record<string, unknown>
: undefined;
const hasEffort = reasoningRecord !== undefined
&& Object.prototype.hasOwnProperty.call(reasoningRecord, "effort");
const callerEffort = reasoningRecord?.effort;
const validCallerEffort = typeof callerEffort === "string" && isCodexReasoningEffort(callerEffort);
const needsDefault = reasoning === undefined || (reasoningRecord !== undefined && !hasEffort);
const shouldForce = defaultEffortMode === "force" && validCallerEffort;
if (!needsDefault && !shouldForce) return clone;
// Picker availability treats an unknown ladder as a wildcard, but runtime
// injection stays fail-closed until this concrete target advertises support.
//
Expand Down
25 changes: 23 additions & 2 deletions src/combos/types.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { isCodexReasoningEffort } from "../reasoning-effort";
import { SUPPORTED_NATIVE_OPENAI_SLUGS } from "../codex/catalog/native-models";
import type { OcxComboConfig, OcxComboDefaultEffort, OcxComboReasoningEffortMode, OcxComboStrategy, OcxComboTarget, OcxProviderConfig } from "../types";
import type { OcxComboConfig, OcxComboDefaultEffort, OcxComboDefaultEffortMode, OcxComboReasoningEffortMode, OcxComboStrategy, OcxComboTarget, OcxProviderConfig } from "../types";
import { COMBO_NAMESPACE, isValidComboId, targetKey } from "./identifiers";

export const COMBO_DEFAULT_WAIT_FOR_COOLDOWN_MS = 0;
Expand All @@ -26,6 +26,8 @@ export interface NormalizedComboConfig {
cooldownMs?: number;
waitForCooldownMs: number;
defaultEffort: OcxComboDefaultEffort | null;
/** Client-precedence policy; `fallback` preserves legacy behavior. */
defaultEffortMode: OcxComboDefaultEffortMode;
/** Picker-ladder derivation policy; `strict` preserves the legacy intersection rule. */
reasoningEffortMode: OcxComboReasoningEffortMode;
/** Disable image input; `auto` preserves the intersection derived from all targets. */
Expand Down Expand Up @@ -167,6 +169,21 @@ export function comboConfigIssues(
message: "defaultEffort must be one of: low, medium, high, xhigh, max, ultra",
});
}
if (body.defaultEffortMode !== undefined
&& body.defaultEffortMode !== "fallback"
&& body.defaultEffortMode !== "force") {
issues.push({
path: ["defaultEffortMode"],
message: 'defaultEffortMode must be "fallback" or "force"',
});
}
if (body.defaultEffortMode === "force"
&& (typeof body.defaultEffort !== "string" || !isCodexReasoningEffort(body.defaultEffort))) {
issues.push({
path: ["defaultEffort"],
message: "defaultEffort is required when defaultEffortMode is force",
});
}
if (body.imageInput !== undefined && body.imageInput !== "auto" && body.imageInput !== "disabled") {
issues.push({ path: ["imageInput"], message: 'imageInput must be "auto" or "disabled"' });
}
Expand Down Expand Up @@ -293,12 +310,16 @@ export function comboConfigError(
export function normalizeComboConfig(raw: OcxComboConfig): NormalizedComboConfig {
const alias = typeof raw.alias === "string" ? raw.alias.trim() : "";
const displayName = typeof raw.displayName === "string" ? raw.displayName.trim() : "";
const defaultEffort = typeof raw.defaultEffort === "string" && isCodexReasoningEffort(raw.defaultEffort)
? raw.defaultEffort
: null;
return {
strategy: raw.strategy ?? "failover",
stickyLimit: raw.stickyLimit ?? 1,
cooldownMs: raw.cooldownMs,
waitForCooldownMs: raw.waitForCooldownMs ?? COMBO_DEFAULT_WAIT_FOR_COOLDOWN_MS,
defaultEffort: raw.defaultEffort ?? null,
defaultEffort,
defaultEffortMode: raw.defaultEffortMode === "force" && defaultEffort !== null ? "force" : "fallback",
reasoningEffortMode: raw.reasoningEffortMode === "adaptive" ? "adaptive" : "strict",
imageInput: raw.imageInput === "disabled" ? "disabled" : "auto",
alias: alias || null,
Expand Down
4 changes: 3 additions & 1 deletion src/server/chat-completions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,9 @@ async function handleChatCompletionsWithBudget(
if (chatBody.tools !== undefined) parts.push(JSON.stringify(chatBody.tools));
logCtx.usageLogInputTokens = Math.max(1, estimateTokens(parts.join("\n"), requestedModel));
}
if (!effortRow && isNativeChatRouteEligible(route, chatBody)) chatNativeRoute = route;
// Combos must enter the Responses routing path so child selection, forced default
// effort, failover, and per-attempt telemetry are applied before any native Chat send.
if (!route.combo && !effortRow && isNativeChatRouteEligible(route, chatBody)) chatNativeRoute = route;
} catch (err) {
if (err instanceof UnknownRoutingPolicyError) {
logCtx.requestedModel = requestedModel;
Expand Down
11 changes: 10 additions & 1 deletion src/server/management/combo-routes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -80,17 +80,20 @@ function sparseComboConfig<T extends {
waitForCooldownMs?: number;
imageInput?: "auto" | "disabled";
reasoningEffortMode?: "strict" | "adaptive";
}>(combo: T): Omit<T, "cooldownMs" | "waitForCooldownMs" | "imageInput" | "reasoningEffortMode"> & {
defaultEffortMode?: "fallback" | "force";
}>(combo: T): Omit<T, "cooldownMs" | "waitForCooldownMs" | "imageInput" | "reasoningEffortMode" | "defaultEffortMode"> & {
cooldownMs?: number;
waitForCooldownMs?: number;
imageInput?: "disabled";
reasoningEffortMode?: "adaptive";
defaultEffortMode?: "force";
} {
const {
cooldownMs,
waitForCooldownMs,
imageInput,
reasoningEffortMode,
defaultEffortMode,
...rest
} = combo;
return {
Expand All @@ -101,6 +104,7 @@ function sparseComboConfig<T extends {
: {}),
...(imageInput === "disabled" ? { imageInput: "disabled" as const } : {}),
...(reasoningEffortMode === "adaptive" ? { reasoningEffortMode: "adaptive" as const } : {}),
...(defaultEffortMode === "force" ? { defaultEffortMode: "force" as const } : {}),
};
}

Expand Down Expand Up @@ -170,6 +174,11 @@ export async function handleComboRoutes(ctx: ManagementContext): Promise<Respons
...(!Object.hasOwn(requestedCombo, "waitForCooldownMs") && previous?.waitForCooldownMs !== undefined
? { waitForCooldownMs: previous.waitForCooldownMs }
: {}),
// The dashboard does not expose this advanced CLI/API policy. Preserve it when
// a GUI round-trip omits the field instead of silently downgrading to fallback.
...(!Object.hasOwn(requestedCombo, "defaultEffortMode") && previous?.defaultEffortMode !== undefined
? { defaultEffortMode: previous.defaultEffortMode }
: {}),
};
const error = comboConfigError(id, effectiveCombo, config.providers, {
requireEnabledTarget: true,
Expand Down
26 changes: 26 additions & 0 deletions src/server/responses/core.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import {
describeOutboundBodyRefusal,
} from "./outbound-body-guard";
import { nativeContextLimits } from "../../codex/catalog";
import { isDeclaredReasoningEffort } from "../../reasoning-effort";
import { describeUpstreamConnectFailure } from "./upstream-error";
import type { CodexWsQuotaObserver } from "./codex-ws-metadata";
import { applyAccountQuotaFromUpstreamHeaders as applyCapturedCodexQuota } from "../../codex/quota";
Expand Down Expand Up @@ -2678,6 +2679,25 @@ export async function handleComboResponses(
// adoption below must never replace it with a concrete child route trace.
logCtx.routeDecision = comboRouteDecisionTrace(config, comboId, pick, requestedModel);

const originalReasoning = body && typeof body === "object" && !Array.isArray(body)
? (body as { reasoning?: unknown }).reasoning
: undefined;
const originalRequestedEffortValue = originalReasoning && typeof originalReasoning === "object" && !Array.isArray(originalReasoning)
? (originalReasoning as { effort?: unknown }).effort
: undefined;
const originalRequestedEffort = typeof originalRequestedEffortValue === "string"
&& isDeclaredReasoningEffort(originalRequestedEffortValue)
? originalRequestedEffortValue
: undefined;
const restoreOriginalRequestedEffort = (childLog: RequestLogContext): void => {
if (originalRequestedEffort === undefined) return;
const normalizedRequestedEffort = childLog.requestedEffort;
const transitionIndex = normalizedRequestedEffort?.indexOf("->") ?? -1;
childLog.requestedEffort = transitionIndex >= 0
? `${originalRequestedEffort}${normalizedRequestedEffort!.slice(transitionIndex)}`
: originalRequestedEffort;
recordAttemptRequestedEffort(childLog);
};
let lastFailure: Response | null = null;
while (pick) {
if (options.abortSignal?.aborted) return clientCancelledResponse();
Expand All @@ -2693,6 +2713,7 @@ export async function handleComboResponses(
pick.target,
comboDefaultEffort(config, comboId),
supportedLadderFor({ provider: targetRoute.provider, modelId: targetRoute.modelId }),
combo.defaultEffortMode,
);
const childHeaders = buildComboChildHeaders(req.headers);
const childRequest = new Request(req.url, {
Expand All @@ -2710,6 +2731,10 @@ export async function handleComboResponses(
config.providers[pick.target.provider]!.adapter,
);
childLog.activeAttempt = attempt;
if (originalRequestedEffort !== undefined) {
childLog.requestedEffort = originalRequestedEffort;
recordAttemptRequestedEffort(childLog);
}
let attemptRetained = false;
const retainCancelledAttempt = (): void => {
if (attemptRetained) return;
Expand Down Expand Up @@ -2781,6 +2806,7 @@ export async function handleComboResponses(
onNativePassthroughCancel: callbackGate.onCancel,
onResponseComplete: callbackGate.onResponseComplete,
});
restoreOriginalRequestedEffort(childLog);
} catch (error) {
callbackGate.discard();
if (options.abortSignal?.aborted) {
Expand Down
1 change: 1 addition & 0 deletions src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ export type {
OcxAccountPoolQuotaWindow,
OcxComboStrategy,
OcxComboDefaultEffort,
OcxComboDefaultEffortMode,
OcxComboReasoningEffortMode,
OcxComboTarget,
OcxComboConfig,
Expand Down
Loading
Loading