Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 64 additions & 2 deletions harness/clients/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,67 @@ MAX_CTX=32768 MAX_TOKENS=512 \
harness/clients/run_codex.sh
```

## GPU selection

All launchers inherit `CUDA_VISIBLE_DEVICES` and `HIP_VISIBLE_DEVICES`. Native
Lucebox runs also accept `TARGET_DEVICE` and `DRAFT_DEVICE`; when the latter is
omitted it follows `TARGET_DEVICE`. Device numbers use the runtime-visible
namespace, so exposing one physical GPU makes it device zero inside the server:

```bash
# NVIDIA GPU 0 with a CUDA build.
CUDA_VISIBLE_DEVICES=0 \
DFLASH_SERVER_BIN=server/build-cuda/dflash_server \
TARGET_DEVICE=cuda:0 \
harness/clients/run_codex.sh

# Physical HIP GPU 1, exposed as hip:0 to a HIP build.
HIP_VISIBLE_DEVICES=1 \
DFLASH_SERVER_BIN=server/build-hip/dflash_server \
TARGET_DEVICE=hip:0 \
harness/clients/run_codex.sh
```

CUDA and HIP servers are separate build artifacts. A host with Strix Halo
(`gfx1151`) and an R9700 (`gfx1201`) can use one dual-architecture HIP build:

```bash
cmake -S server -B server/build-hip \
-DCMAKE_BUILD_TYPE=Release \
-DDFLASH27B_GPU_BACKEND=hip \
-DDFLASH27B_HIP_ARCHITECTURES='gfx1151;gfx1201' \
-DDFLASH27B_HIP_SM80_EQUIV=ON
cmake --build server/build-hip --target dflash_server -j"$(nproc)"
```

Select a server binary and matching visibility variable together. The launcher
prints resolved placement before startup and uses `nvidia-smi` or `rocm-smi`
for the matching backend in its final report.

## Interactive terminal clients

The default launcher behavior remains a deterministic one-shot compatibility
test. Set `HARNESS_INTERACTIVE=1` to attach the real client TUI to the terminal
while the launcher manages the Lucebox server:

```bash
HARNESS_INTERACTIVE=1 harness/clients/run_claude_code.sh
HARNESS_INTERACTIVE=1 harness/clients/run_codex.sh
HARNESS_INTERACTIVE=1 harness/clients/run_opencode.sh
HARNESS_INTERACTIVE=1 harness/clients/run_hermes.sh
HARNESS_INTERACTIVE=1 harness/clients/run_pi.sh
HARNESS_INTERACTIVE=1 harness/clients/run_openclaw.sh
```

`INTERACTIVE_PROMPT` supplies an optional first message where the client
supports it. Interactive client state and sessions persist under
`.harness-work/interactive/<client>`. Exit the client or press Ctrl+C to stop
the launcher-managed server. One-shot client timeouts do not apply to a TUI.
Set `HARNESS_PROGRESS=0` to suppress launcher progress and heartbeat messages.

Open WebUI is already interactive through its browser UI; its harness scripts
remain deterministic HTTP probes.

The C++ server is expected to handle the same client protocol shapes covered by
these launchers and probes: OpenAI Chat Completions, streaming chunks, tool
metadata, OpenAI Responses for Codex, Anthropic Messages for Claude Code, and
Expand Down Expand Up @@ -128,5 +189,6 @@ Responses requests can be compared too.
- `common.sh` contains the shared server startup logic.
- `run_openwebui_tools.sh` supports `OPENWEBUI_FUNCTION_CALLING=default` and
`OPENWEBUI_FUNCTION_CALLING=native`.
- Every launcher redirects stdin from `/dev/null`; this prevents SSH input from
being accidentally treated as a user prompt by interactive clients.
- One-shot launchers redirect stdin from `/dev/null`; this prevents SSH input
from being accidentally treated as a user prompt. `HARNESS_INTERACTIVE=1`
deliberately keeps the selected terminal client attached to the TTY.
111 changes: 109 additions & 2 deletions harness/clients/common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,14 @@ LLAMA_CACHE_TYPE_K="${LLAMA_CACHE_TYPE_K:-${CACHE_TYPE_K:-q8_0}}"
LLAMA_CACHE_TYPE_V="${LLAMA_CACHE_TYPE_V:-${CACHE_TYPE_V:-q8_0}}"
MAX_TOKENS="${MAX_TOKENS:-2048}"
EXTRA_SERVER_ARGS="${EXTRA_SERVER_ARGS:-}"
TARGET_DEVICE="${TARGET_DEVICE:-}"
DRAFT_DEVICE="${DRAFT_DEVICE:-${TARGET_DEVICE:-}}"
HARNESS_INTERACTIVE="${HARNESS_INTERACTIVE:-0}"
INTERACTIVE_PROMPT="${INTERACTIVE_PROMPT:-}"
if [[ "$HARNESS_INTERACTIVE" != "0" && "$HARNESS_INTERACTIVE" != "1" ]]; then
echo "HARNESS_INTERACTIVE must be 0 or 1" >&2
exit 2
fi

MODEL_ID="${MODEL_ID:-luce-dflash}"
API_KEY="${API_KEY:-sk-lucebox}"
Expand All @@ -81,6 +89,37 @@ SERVER_LOG="$LOG_DIR/server.log"

mkdir -p "$LOG_DIR"

# Keep progress attached to the original terminal even while one-shot client
# stdout/stderr is redirected to its result file.
exec 3>&2
HARNESS_PROGRESS="${HARNESS_PROGRESS:-1}"

progress() {
if [[ "$HARNESS_PROGRESS" != "0" ]]; then
printf '[harness] %s\n' "$*" >&3
fi
}

client_home() {
local client="$1"
if [[ "$HARNESS_INTERACTIVE" == "1" ]]; then
printf '%s/interactive/%s\n' "$CLIENT_WORK_DIR" "$client"
else
printf '%s/%s-home\n' "$LOG_DIR" "$client"
fi
}

run_interactive_client() {
local label="$1"
local client_out="$2"
shift 2
progress "opening interactive $label; exit the client to stop the server"
"$@"
local rc=$?
printf 'Interactive %s terminal output was not captured.\n' "$label" > "$client_out"
return "$rc"
}

require_client_binary() {
local label="$1"
local path="$2"
Expand Down Expand Up @@ -118,18 +157,53 @@ run_with_timeout() {
echo "client timeout must be a non-negative integer (seconds; 0 disables it)" >&2
return 2
fi
local command_name
command_name="$(basename "$1")"
if [[ "$command_name" == "env" ]]; then
local arg
for arg in "${@:2}"; do
if [[ "$arg" != *=* ]]; then
command_name="$(basename "$arg")"
break
fi
done
fi
local timeout_label="${timeout_seconds}s"
if [[ "$timeout_seconds" == "0" ]]; then
timeout_label="disabled"
fi
progress "running $command_name (timeout: $timeout_label; logs: $LOG_DIR)"

local started_at heartbeat_pid rc elapsed
started_at="$(date +%s)"
(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: If the OpenClaw config preflight fails or times out, set -e exits run_with_timeout before the heartbeat cleanup runs, leaving its background loop alive and continuing to emit progress. Install heartbeat cleanup in an unconditional trap or run the wrapped command through an errexit-safe conditional.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At harness/clients/common.sh, line 179:

<comment>If the OpenClaw config preflight fails or times out, `set -e` exits `run_with_timeout` before the heartbeat cleanup runs, leaving its background loop alive and continuing to emit progress. Install heartbeat cleanup in an unconditional trap or run the wrapped command through an errexit-safe conditional.</comment>

<file context>
@@ -118,18 +157,53 @@ run_with_timeout() {
+
+  local started_at heartbeat_pid rc elapsed
+  started_at="$(date +%s)"
+  (
+    while sleep 10; do
+      elapsed=$(( $(date +%s) - started_at ))
</file context>

while sleep 10; do
elapsed=$(( $(date +%s) - started_at ))
progress "$command_name still running (${elapsed}s elapsed)"
done
) &
heartbeat_pid=$!

if [[ "$timeout_seconds" == "0" ]]; then
"$@"
rc=$?
else
timeout "${timeout_seconds}s" "$@"
rc=$?
fi
kill "$heartbeat_pid" 2>/dev/null || true
wait "$heartbeat_pid" 2>/dev/null || true
elapsed=$(( $(date +%s) - started_at ))
progress "$command_name finished (rc=$rc, ${elapsed}s elapsed)"
return "$rc"
}

draft_enabled() {
[[ -n "${DRAFT:-}" && "$DRAFT" != "none" && "$DRAFT" != "off" && "$DRAFT" != "0" ]]
}

start_lucebox_server() {
progress "starting $MODEL_SERVER server (log: $SERVER_LOG)"
if [[ "$MODEL_SERVER" == "llamacpp" ]]; then
start_llamacpp_server
return
Expand Down Expand Up @@ -177,17 +251,26 @@ start_dflash_native_server() {
if [[ -n "$FA_WINDOW" ]] && [[ "$FA_WINDOW" != "0" ]]; then
fa_args=(--fa-window "$FA_WINDOW")
fi
local device_args=()
if [[ -n "$TARGET_DEVICE" ]]; then
device_args+=(--target-device "$TARGET_DEVICE")
fi
if [[ -n "$DRAFT_DEVICE" ]]; then
device_args+=(--draft-device "$DRAFT_DEVICE")
fi
# Export KV cache type env vars for the C++ server to pick up (only when
# explicitly requested: the per-axis envs override family defaults).
if [[ -n "$CACHE_TYPE_K" ]]; then export DFLASH27B_KV_K="$CACHE_TYPE_K"; fi
if [[ -n "$CACHE_TYPE_V" ]]; then export DFLASH27B_KV_V="$CACHE_TYPE_V"; fi
progress "server placement: target=${TARGET_DEVICE:-auto:0} draft=${DRAFT_DEVICE:-auto:0} CUDA_VISIBLE_DEVICES=${CUDA_VISIBLE_DEVICES:-unset} HIP_VISIBLE_DEVICES=${HIP_VISIBLE_DEVICES:-unset}"
"$DFLASH_SERVER_BIN" "$TARGET" \
"${draft_args[@]}" \
--host "$HOST" \
--port "$PORT" \
--max-ctx "$MAX_CTX" \
--max-tokens "$MAX_TOKENS" \
--model-name "$MODEL_ID" \
"${device_args[@]}" \
"${ddtree_args[@]}" \
"${fa_args[@]}" \
"${extra_args[@]}" \
Expand Down Expand Up @@ -264,10 +347,15 @@ start_llamacpp_server() {
}

wait_lucebox_server() {
for _ in $(seq 1 300); do
progress "waiting for server health at $BASE_URL/health"
for attempt in $(seq 1 300); do
if curl -fsS "$BASE_URL/health" >/dev/null 2>&1; then
progress "server is healthy"
return 0
fi
if (( attempt % 10 == 0 )); then
progress "server still starting (${attempt}s elapsed; log: $SERVER_LOG)"
fi
sleep 1
if ! kill -0 "$SERVER_PID" 2>/dev/null; then
echo "server exited early; log: $SERVER_LOG" >&2
Expand Down Expand Up @@ -304,5 +392,24 @@ finish_report() {
echo "--- server tail ---"
tail -n 120 "$SERVER_LOG" || true
echo "--- gpu ---"
nvidia-smi --query-gpu=name,memory.used,memory.total,utilization.gpu --format=csv,noheader || true
local resolved_backend="${TARGET_DEVICE%%:*}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When the HIP server uses default or TARGET_DEVICE=auto:0 placement, finish_report invokes nvidia-smi because auto:0 is never resolved to HIP. Resolve auto from the actual server backend before selecting the GPU tool, or make startup expose the resolved backend.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At harness/clients/common.sh, line 395:

<comment>When the HIP server uses default or `TARGET_DEVICE=auto:0` placement, `finish_report` invokes `nvidia-smi` because `auto:0` is never resolved to HIP. Resolve `auto` from the actual server backend before selecting the GPU tool, or make startup expose the resolved backend.</comment>

<file context>
@@ -304,5 +392,24 @@ finish_report() {
   tail -n 120 "$SERVER_LOG" || true
   echo "--- gpu ---"
-  nvidia-smi --query-gpu=name,memory.used,memory.total,utilization.gpu --format=csv,noheader || true
+  local resolved_backend="${TARGET_DEVICE%%:*}"
+  if [[ -z "$TARGET_DEVICE" ]] && grep -Eq 'target_device[[:space:]]*=[[:space:]]*hip:' "$SERVER_LOG"; then
+    resolved_backend="hip"
</file context>

if [[ -z "$TARGET_DEVICE" ]] && grep -Eq 'target_device[[:space:]]*=[[:space:]]*hip:' "$SERVER_LOG"; then
resolved_backend="hip"
elif [[ -z "$TARGET_DEVICE" ]] && grep -Eq 'target_device[[:space:]]*=[[:space:]]*cuda:' "$SERVER_LOG"; then
resolved_backend="cuda"
fi
if [[ "$resolved_backend" == "hip" ]]; then
if command -v rocm-smi >/dev/null 2>&1; then
local rocm_device_args=()
if [[ "${HIP_VISIBLE_DEVICES:-}" =~ ^[0-9]+$ ]]; then

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The rocm-smi --device selection only handles a single-integer HIP_VISIBLE_DEVICES. Comma-separated lists (documented for dual-GPU HIP runs) fail the ^[0-9]+$ regex, so --device is dropped and rocm-smi reports all devices instead of the selected slot. Consider splitting the list on commas and passing the mapped runtime-index list, or drop --device entirely since HIP_VISIBLE_DEVICES already scopes visibility.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At harness/clients/common.sh, line 404:

<comment>The rocm-smi `--device` selection only handles a single-integer HIP_VISIBLE_DEVICES. Comma-separated lists (documented for dual-GPU HIP runs) fail the `^[0-9]+$` regex, so --device is dropped and rocm-smi reports all devices instead of the selected slot. Consider splitting the list on commas and passing the mapped runtime-index list, or drop --device entirely since HIP_VISIBLE_DEVICES already scopes visibility.</comment>

<file context>
@@ -304,5 +392,24 @@ finish_report() {
+  if [[ "$resolved_backend" == "hip" ]]; then
+    if command -v rocm-smi >/dev/null 2>&1; then
+      local rocm_device_args=()
+      if [[ "${HIP_VISIBLE_DEVICES:-}" =~ ^[0-9]+$ ]]; then
+        rocm_device_args=(--device "$HIP_VISIBLE_DEVICES")
+      fi
</file context>

rocm_device_args=(--device "$HIP_VISIBLE_DEVICES")
fi
rocm-smi "${rocm_device_args[@]}" --showproductname --showuse --showmeminfo vram 2>/dev/null || true
else
echo "rocm-smi not found"
fi
else
nvidia-smi --query-gpu=name,memory.used,memory.total,utilization.gpu --format=csv,noheader 2>/dev/null || \
echo "nvidia-smi unavailable"
fi
}
47 changes: 28 additions & 19 deletions harness/clients/run_claude_code.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ source "$SCRIPT_DIR/common.sh"
CLIENT_OUT="$LOG_DIR/claude-code.out"
CLAUDE_BIN="${CLAUDE_BIN:-$CLIENT_WORK_DIR/clients/claude_code/npm/bin/claude}"
require_client_binary "Claude Code" "$CLAUDE_BIN" "claude_code" "CLAUDE_BIN"
HOME_DIR="$LOG_DIR/claude-home"
HOME_DIR="$(client_home claude)"
mkdir -p "$HOME_DIR"

start_lucebox_server
Expand Down Expand Up @@ -57,25 +57,34 @@ if [[ -n "${PFLASH_SESSION_ID:-}" ]]; then
echo "[run_claude_code] session-inject proxy up on $CLIENT_BASE_URL (session=$PFLASH_SESSION_ID)"
fi

claude_env=(
"HOME=$HOME_DIR"
"ANTHROPIC_API_KEY=$API_KEY"
"ANTHROPIC_BASE_URL=$CLIENT_BASE_URL"
"CLAUDE_CODE_API_BASE_URL=$CLIENT_BASE_URL"
"CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1"
"CLAUDE_CODE_DISABLE_TELEMETRY=1"
"CLAUDE_CODE_DISABLE_NONSTREAMING_FALLBACK=1"
)
set +e
run_with_timeout "$CLAUDE_TIMEOUT" env \
HOME="$HOME_DIR" \
ANTHROPIC_API_KEY="$API_KEY" \
ANTHROPIC_BASE_URL="$CLIENT_BASE_URL" \
CLAUDE_CODE_API_BASE_URL="$CLIENT_BASE_URL" \
CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 \
CLAUDE_CODE_DISABLE_TELEMETRY=1 \
CLAUDE_CODE_DISABLE_NONSTREAMING_FALLBACK=1 \
"$CLAUDE_BIN" \
--print \
--output-format json \
--model "$MODEL_ID" \
--tools "$CLAUDE_TOOLS" \
--permission-mode dontAsk \
--no-session-persistence \
"$PROMPT" \
< /dev/null > "$CLIENT_OUT" 2>&1
RC=$?
if [[ "$HARNESS_INTERACTIVE" == "1" ]]; then
claude_cmd=("$CLAUDE_BIN" --model "$MODEL_ID" --tools "$CLAUDE_TOOLS")
if [[ -n "$INTERACTIVE_PROMPT" ]]; then claude_cmd+=("$INTERACTIVE_PROMPT"); fi
run_interactive_client "Claude Code" "$CLIENT_OUT" env "${claude_env[@]}" "${claude_cmd[@]}"
RC=$?
else
run_with_timeout "$CLAUDE_TIMEOUT" env "${claude_env[@]}" \
"$CLAUDE_BIN" \
--print \
--output-format json \
--model "$MODEL_ID" \
--tools "$CLAUDE_TOOLS" \
--permission-mode dontAsk \
--no-session-persistence \
"$PROMPT" \
< /dev/null > "$CLIENT_OUT" 2>&1
RC=$?
fi
set -e

if [[ -n "$PROXY_PID" ]] && kill -0 "$PROXY_PID" 2>/dev/null; then
Expand Down
37 changes: 22 additions & 15 deletions harness/clients/run_codex.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ CLIENT_OUT="$LOG_DIR/codex.out"
LAST_MSG="$LOG_DIR/codex-last-message.txt"
CODEX_BIN="${CODEX_BIN:-$CLIENT_WORK_DIR/clients/codex/npm/bin/codex}"
require_client_binary "Codex" "$CODEX_BIN" "codex" "CODEX_BIN"
CODEX_HOME_DIR="$LOG_DIR/codex-home"
CODEX_HOME_DIR="$(client_home codex)"
CODEX_SANDBOX="${CODEX_SANDBOX:-danger-full-access}"
CODEX_WIRE_API="${CODEX_WIRE_API:-responses}"
mkdir -p "$CODEX_HOME_DIR"
Expand All @@ -38,22 +38,29 @@ start_lucebox_server
trap stop_lucebox_server EXIT
wait_lucebox_server

codex_env=("HOME=$CODEX_HOME_DIR" "CODEX_HOME=$CODEX_HOME_DIR" "OPENAI_API_KEY=$API_KEY")
set +e
run_with_timeout "$CODEX_TIMEOUT" env \
HOME="$CODEX_HOME_DIR" \
CODEX_HOME="$CODEX_HOME_DIR" \
OPENAI_API_KEY="$API_KEY" \
"$CODEX_BIN" exec \
--skip-git-repo-check \
--sandbox "$CODEX_SANDBOX" \
--model "$MODEL_ID" \
--json \
--output-last-message "$LAST_MSG" \
"$PROMPT" \
< /dev/null > "$CLIENT_OUT" 2>&1
RC=$?
if [[ "$HARNESS_INTERACTIVE" == "1" ]]; then
codex_cmd=("$CODEX_BIN" --cd "$REPO_DIR" --sandbox "$CODEX_SANDBOX" --model "$MODEL_ID")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The interactive Codex branch drops --skip-git-repo-check, which the one-shot path explicitly uses, so a codex --cd <dir> TUI can fail to open when that directory is not a git repo. Add the flag to the interactive command for parity with the non-interactive launcher.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At harness/clients/run_codex.sh, line 44:

<comment>The interactive Codex branch drops `--skip-git-repo-check`, which the one-shot path explicitly uses, so a `codex --cd <dir>` TUI can fail to open when that directory is not a git repo. Add the flag to the interactive command for parity with the non-interactive launcher.</comment>

<file context>
@@ -38,22 +38,29 @@ start_lucebox_server
-  < /dev/null > "$CLIENT_OUT" 2>&1
-RC=$?
+if [[ "$HARNESS_INTERACTIVE" == "1" ]]; then
+  codex_cmd=("$CODEX_BIN" --cd "$REPO_DIR" --sandbox "$CODEX_SANDBOX" --model "$MODEL_ID")
+  if [[ -n "$INTERACTIVE_PROMPT" ]]; then codex_cmd+=("$INTERACTIVE_PROMPT"); fi
+  run_interactive_client "Codex" "$CLIENT_OUT" env "${codex_env[@]}" "${codex_cmd[@]}"
</file context>
Suggested change
codex_cmd=("$CODEX_BIN" --cd "$REPO_DIR" --sandbox "$CODEX_SANDBOX" --model "$MODEL_ID")
codex_cmd=("$CODEX_BIN" --cd "$REPO_DIR" --skip-git-repo-check --sandbox "$CODEX_SANDBOX" --model "$MODEL_ID")

if [[ -n "$INTERACTIVE_PROMPT" ]]; then codex_cmd+=("$INTERACTIVE_PROMPT"); fi
run_interactive_client "Codex" "$CLIENT_OUT" env "${codex_env[@]}" "${codex_cmd[@]}"
RC=$?
else
run_with_timeout "$CODEX_TIMEOUT" env "${codex_env[@]}" \
"$CODEX_BIN" exec \
--skip-git-repo-check \
--sandbox "$CODEX_SANDBOX" \
--model "$MODEL_ID" \
--json \
--output-last-message "$LAST_MSG" \
"$PROMPT" \
< /dev/null > "$CLIENT_OUT" 2>&1
RC=$?
fi
set -e

cat "$LAST_MSG" >> "$CLIENT_OUT" 2>/dev/null || true
if [[ "$HARNESS_INTERACTIVE" == "0" ]]; then
cat "$LAST_MSG" >> "$CLIENT_OUT" 2>/dev/null || true
fi
finish_report "$CLIENT_OUT" "$RC"
exit "$RC"
Loading
Loading