Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 5 additions & 38 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,43 +16,11 @@ jobs:
fetch-depth: 0 # 拉取完整提交历史
fetch-tags: true # 关键:拉取所有标签

- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: '1.23' # 请根据需要选择合适的 Go 版本

- name: Install Flatbuffers
run: |
wget -q https://github.com/google/flatbuffers/releases/download/v24.12.23/Linux.flatc.binary.clang++-18.zip
unzip Linux.flatc.binary.clang++-18.zip
sudo cp flatc /usr/local/bin
rm flatc Linux.flatc.binary.clang++-18.zip

- name: Verify Go Version
run: go version

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.13'

- name: Verify Python Version
run: python --version

- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
pip install build

- name: Install Docker
uses: docker/setup-buildx-action@v2

- name: Install Dependencies & Compile Files
- name: Install system dependencies & build release
run: |
sudo apt update
sudo apt install portaudio19-dev libx11-dev libxtst-dev
# curl -fsSL https://get.docker.com -o get-docker.sh
# sudo sh get-docker.sh
make
make build-release

- name: Create GitHub Release
id: create_release
Expand All @@ -63,8 +31,7 @@ jobs:
body: |
SPEAR release version ${{ github.ref_name }}.
files: |
./bin/spearlet
./sdk/python/dist/spearlet-*.whl
./sdk/python/dist/spearlet-*.tar.gz
./target/release/spearlet
./target/release/sms
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # 使用 GitHub 提供的令牌
26 changes: 5 additions & 21 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ tokio-util = "0.7"

# gRPC framework / gRPC框架
tonic = "0.12"
tonic-web = "0.12"
prost = "0.13"
prost-types = "0.13"

Expand Down Expand Up @@ -89,6 +88,7 @@ evmap = { version = "10.0", optional = true }

# System calls / 系统调用
libc = "0.2"
spear-ssf = { path = "sdk/rust/crates/spear-ssf" }

# MD5 hashing / MD5哈希
md5 = "0.7"
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -563,7 +563,7 @@ SAMPLES_CFLAGS ?=
SAMPLES_JS_DIR ?= samples/wasm-js
SAMPLES_JS_BUILD ?= $(SAMPLES_BUILD)/js
JS_WASM_PREFIX ?= js-
JS_SAMPLES ?= chat_completion chat_completion_tool_sum router_filter_keyword user_stream_echo
JS_SAMPLES ?= chat_completion chat_completion_tool_sum router_filter_keyword user_stream_echo user_stream_chat_completion
BUILD_JS_SAMPLES ?= 1
SAMPLES_RUST_DIR ?= $(SAMPLES_JS_DIR)
RUST_SAMPLES ?= $(JS_SAMPLES)
Expand Down
8 changes: 4 additions & 4 deletions assets/console/main.js

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions deploy/docker/sms/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,9 @@ COPY proto/ proto/
COPY assets/ assets/
COPY src/ src/

# Local path dependencies (SDK crates) / 本地 path 依赖(SDK crates)
COPY sdk/rust/crates/spear-ssf/ sdk/rust/crates/spear-ssf/

COPY --from=web_builder /work/assets/admin assets/admin
COPY --from=web_builder /work/assets/console assets/console

Expand Down
3 changes: 3 additions & 0 deletions deploy/docker/spear/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,9 @@ COPY proto/ proto/
COPY assets/ assets/
COPY src/ src/

# Local path dependencies (SDK crates) / 本地 path 依赖(SDK crates)
COPY sdk/rust/crates/spear-ssf/ sdk/rust/crates/spear-ssf/

COPY --from=web_builder /work/assets/admin assets/admin
COPY --from=web_builder /work/assets/console assets/console

Expand Down
3 changes: 3 additions & 0 deletions deploy/docker/spearlet/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,9 @@ COPY proto/ proto/
COPY assets/ assets/
COPY src/ src/

# Local path dependencies (SDK crates) / 本地 path 依赖(SDK crates)
COPY sdk/rust/crates/spear-ssf/ sdk/rust/crates/spear-ssf/

RUN cargo build --release --locked --bin spearlet

FROM ${DEBIAN_IMAGE} AS runtime
Expand Down
20 changes: 19 additions & 1 deletion docs/DEAD_CODE_CLEANUP-en.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,24 @@

This document records the dead code cleanup work performed on the spear-next project, aimed at improving code quality, reducing maintenance burden, and optimizing compilation performance.

## 2026-05 Cleanup (Repository Hygiene)

### Changes
- Removed unused direct dependency `tonic-web` from `Cargo.toml`.
- Updated GitHub release workflow to remove obsolete Go/Python/FlatBuffers steps and to upload the actual build outputs:
- `target/release/spearlet`
- `target/release/sms`
- Removed unused placement decision tracking state (kept `decision_id` as a lightweight correlation token).
- Removed an unused duplicate `resolve_spearlet_ws_url` implementation from the endpoint gateway.
- Cleaned MCP registry sync cursor handling to avoid redundant initialization.
- Reduced errno exports to the minimal set actually used by host APIs (and kept test-only aliases behind `cfg(test)`).
- Removed redundant client initializations and obvious unused variables in tests/examples.

### Verification
- `cargo check`: ✅ no warnings
- `make build`: ✅ passed
- `make test`: ✅ passed

## Cleanup Details

### 1. Unused Import Cleanup
Expand Down Expand Up @@ -96,4 +114,4 @@ This dead code cleanup work successfully:

**Cleanup Date**: January 2024
**Performed By**: AI Assistant
**Verification Status**: ✅ Verified successful compilation
**Verification Status**: ✅ Verified successful compilation
20 changes: 19 additions & 1 deletion docs/DEAD_CODE_CLEANUP-zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,24 @@

This document records the dead code cleanup work performed on the spear-next project, aimed at improving code quality, reducing maintenance burden, and optimizing compilation performance.

## 2026-05 清理(仓库卫生) / 2026-05 Cleanup (Repository Hygiene)

### 变更 / Changes
- 从 `Cargo.toml` 中移除了未使用的直接依赖 `tonic-web`。
- 更新 GitHub Release 工作流,移除历史遗留的 Go/Python/FlatBuffers 步骤,并改为上传实际构建产物:
- `target/release/spearlet`
- `target/release/sms`
- 移除了未使用的 placement 决策记录状态(保留 `decision_id` 作为轻量级关联标识)。
- 删除了 endpoint gateway 内部未使用且重复的 `resolve_spearlet_ws_url` 实现。
- 清理 MCP registry sync 的 cursor 初始化逻辑,避免冗余赋值。
- 将 errno 导出收敛到 Host API 实际使用的最小集合(测试用别名放在 `cfg(test)` 下)。
- 删除 tests/examples 中明显冗余的 client 初始化与未使用变量。

### 验证 / Verification
- `cargo check`: ✅ 无警告
- `make build`: ✅ 通过
- `make test`: ✅ 通过

## 清理内容 / Cleanup Details

### 1. 未使用的导入清理 / Unused Import Cleanup
Expand Down Expand Up @@ -104,4 +122,4 @@ This dead code cleanup successfully:

**清理日期 / Cleanup Date**: 2024年1月
**执行者 / Performed By**: AI Assistant
**验证状态 / Verification Status**: ✅ 已验证编译成功
**验证状态 / Verification Status**: ✅ 已验证编译成功
20 changes: 18 additions & 2 deletions docs/api/spear-hostcall/wasm-user-stream-bridge-en.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,11 +154,27 @@ Current implementation requires only a valid SSF v1 header (magic/version/header

Recommended v1 conventions:

- `1 = CTRL`:
- Control frames (e.g. OPEN/keepalive).
- `meta`: JSON UTF-8 (optional).
- `data`: usually empty.
- `2 = DATA`:
- `meta`: optional (per-chunk overrides; recommended to keep empty for performance).
- Data frames (text/binary payload).
- `meta`: JSON UTF-8 (optional).
- `data`: raw bytes (audio/video/text or any binary).
- `3 = COMMIT`:
- Marks the end of a user input segment (application-level).
- `meta`: JSON UTF-8 (optional).
- `data`: usually empty.

Other message types (OPEN/COMMIT/CLOSE/ACK/ERROR/etc.) are reserved for future extensions.
Other message types (CLOSE/ACK/ERROR/PING/PONG/etc.) are reserved for future extensions.

### 4.3.1 SDK constants

- C: `sdk/c/include/spear_ssf.h` provides `SPEAR_SSF_MSG_TYPE_CTRL/DATA/COMMIT`.
- Rust (guest): `spear_wasm::ssf::SsfMsgType`.
- JS (web-console): `web-console/src/ssf/index.ts` exports `SsfMsgType` and `encodeSsfV1Frame/decodeSsfV1Frame`.
- JS (Boa runtime): `import * as ssf from "spear/ssf"` exports `MsgType`.

### 4.4 Metadata conventions

Expand Down
15 changes: 15 additions & 0 deletions docs/api/spear-hostcall/wasm-user-stream-bridge-zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,9 +154,17 @@ Body:

推荐的 v1 约定:

- `1 = CTRL`:
- 控制类帧(例如 OPEN/keepalive)。
- `meta`:JSON UTF-8(可选)。
- `data`:通常为空。
- `2 = DATA`:
- `meta`:可选(每 chunk 覆盖;性能考虑建议常为空)。
- `data`:raw bytes(音频/视频/文本或任意二进制)。
- `3 = COMMIT`:
- 标记一次用户输入片段的结束(应用层语义)。
- `meta`:JSON UTF-8(可选)。
- `data`:通常为空。

其他消息类型(OPEN/COMMIT/CLOSE/ACK/ERROR 等)作为后续扩展预留。
- `4 = CLOSE`:
Expand All @@ -168,6 +176,13 @@ Body:
- `7 = PING`,`8 = PONG`:
- 作为应用层保活补充(WS 自带 ping/pong 之外的可选机制)。

### 4.3.1 SDK 常量

- C:`sdk/c/include/spear_ssf.h` 提供 `SPEAR_SSF_MSG_TYPE_CTRL/DATA/COMMIT`。
- Rust(guest):`spear_wasm::ssf::SsfMsgType`。
- JS(web-console):`web-console/src/ssf/index.ts` 导出 `SsfMsgType` 以及 `encodeSsfV1Frame/decodeSsfV1Frame`。
- JS(Boa 运行时):`import * as ssf from "spear/ssf"` 导出 `MsgType`。

### 4.4 元信息约定

v1 的 metadata 编码:**JSON UTF-8**(便于调试与跨语言互通)。
Expand Down
5 changes: 5 additions & 0 deletions docs/helm-deployment-en.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,11 @@ docker build -f deploy/docker/sms/Dockerfile -t <REGISTRY>/spear-sms:<TAG> .
docker build -f deploy/docker/spearlet/Dockerfile -t <REGISTRY>/spear-spearlet:<TAG> .
```

Local path dependency note:

- `spear-next` depends on `spear-ssf` via a local `path` dependency (`sdk/rust/crates/spear-ssf`).
- The Dockerfiles explicitly copy this SDK crate into the build stage. If you add more local `path` dependencies, update the Dockerfiles accordingly.

Cargo registry note:

- The Dockerfiles use a Cargo registry mirror by default (`rsproxy.cn`) to improve reliability in some network environments.
Expand Down
5 changes: 5 additions & 0 deletions docs/helm-deployment-zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,11 @@ docker build -f deploy/docker/sms/Dockerfile -t <REGISTRY>/spear-sms:<TAG> .
docker build -f deploy/docker/spearlet/Dockerfile -t <REGISTRY>/spear-spearlet:<TAG> .
```

本地 path 依赖说明:

- `spear-next` 通过本地 `path` 依赖引用了 `spear-ssf`(`sdk/rust/crates/spear-ssf`)。
- 这些 Dockerfile 在构建阶段会显式拷贝该 SDK crate;如果你新增了更多本地 `path` 依赖,需要同步更新 Dockerfile。

Cargo registry 说明:

- 这些 Dockerfile 默认使用 Cargo 镜像源(`rsproxy.cn`),主要是为了在某些网络环境下提升依赖下载的稳定性。
Expand Down
5 changes: 4 additions & 1 deletion docs/samples-build-guide-en.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@
- Source: `samples/wasm-c/chat_completion.c` (Chat Completions sample)
- Source: `samples/wasm-js/chat_completion/src/main.rs` (Boa JS runner compiled to WASM; runs `entry.mjs` → Chat Completion)
- Source: `samples/wasm-js/chat_completion_tool_sum/src/main.rs` (Boa JS runner compiled to WASM; runs `entry.mjs` → Tool calling)
- Source: `samples/wasm-js/router_filter_keyword/src/main.rs` (Boa JS runner compiled to WASM; runs `entry.mjs` → Router keyword filter)
- Source: `samples/wasm-js/user_stream_echo/src/main.rs` (Boa JS runner compiled to WASM; runs `entry.mjs` → bidirectional user stream echo)
- Source: `samples/wasm-js/user_stream_chat_completion/src/main.rs` (Boa JS runner compiled to WASM; runs `entry.mjs` → user input over user stream → Chat Completion)
- Source: `samples/wasm-c/mic_rtasr.c` (realtime mic → realtime ASR)
- Output: `samples/build/hello.wasm`
- - WASM-JS outputs: `samples/build/js/js-*.wasm`
Expand Down Expand Up @@ -40,7 +43,7 @@ WASM-JS samples:
- Built by `cargo build --release --target wasm32-wasip1`
- Controlled by Makefile vars:
- `BUILD_JS_SAMPLES=0` to skip WASM-JS samples (compat: `BUILD_RUST_SAMPLES=0`)
- `JS_SAMPLES="chat_completion chat_completion_tool_sum"` to select which samples to build (compat: `RUST_SAMPLES=...`)
- `JS_SAMPLES="chat_completion chat_completion_tool_sum router_filter_keyword user_stream_echo user_stream_chat_completion"` to select which samples to build (compat: `RUST_SAMPLES=...`)
- `JS_WASM_PREFIX="js-"` to set the WASM-JS output filename prefix (default `js-`)

## clang usage
Expand Down
5 changes: 4 additions & 1 deletion docs/samples-build-guide-zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
- 源码:`samples/wasm-c/mic_rtasr.c`(实时麦克风→实时ASR示例)
- 源码:`samples/wasm-js/chat_completion/src/main.rs`(Boa JS runner 编译为 WASM;内部执行 `entry.mjs` → Chat Completion)
- 源码:`samples/wasm-js/chat_completion_tool_sum/src/main.rs`(Boa JS runner 编译为 WASM;内部执行 `entry.mjs` → tool calling)
- 源码:`samples/wasm-js/router_filter_keyword/src/main.rs`(Boa JS runner 编译为 WASM;内部执行 `entry.mjs` → Router 关键词过滤)
- 源码:`samples/wasm-js/user_stream_echo/src/main.rs`(Boa JS runner 编译为 WASM;内部执行 `entry.mjs` → 双向 user stream echo)
- 源码:`samples/wasm-js/user_stream_chat_completion/src/main.rs`(Boa JS runner 编译为 WASM;内部执行 `entry.mjs` → user stream 用户输入 → Chat Completion)
- 产物:`samples/build/hello.wasm`
- WASM-JS 产物:`samples/build/js/js-*.wasm`(兼容:`samples/build/rust/*.wasm`)

Expand Down Expand Up @@ -40,7 +43,7 @@ WASM-JS 示例:
- 通过 `cargo build --release --target wasm32-wasip1` 构建
- 可通过 Makefile 变量控制:
- `BUILD_JS_SAMPLES=0` 跳过 WASM-JS 示例构建(兼容:`BUILD_RUST_SAMPLES=0`)
- `JS_SAMPLES="chat_completion chat_completion_tool_sum"` 指定要构建的示例列表(兼容:`RUST_SAMPLES=...`)
- `JS_SAMPLES="chat_completion chat_completion_tool_sum router_filter_keyword user_stream_echo user_stream_chat_completion"` 指定要构建的示例列表(兼容:`RUST_SAMPLES=...`)
- `JS_WASM_PREFIX="js-"` 配置 JS 产物文件名前缀(默认 `js-`)

## clang 使用说明
Expand Down
4 changes: 3 additions & 1 deletion docs/spear-console-overview-en.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,9 @@ SPEAR Console uses the existing execution stream protocol:

1) User selects a Task -> Instance -> Execution in the “Start a chat” dialog.
2) Console calls `POST /api/v1/executions/{execution_id}/streams/session` (same-origin) to obtain `ws_url`.
3) Console connects to `ws_url` via WebSocket and exchanges SSF frames (stream id `1`, text frames use `msgType=2`).
3) Console connects to `ws_url` via WebSocket and exchanges SSF frames.
- On `open`, Console sends an initial SSF v1 CTRL frame (`msgType=1`, empty data) to establish the stream (`stream_id=1`) without waiting for the first user input.
- User input is sent as one or more DATA frames (`msgType=2`), followed by a COMMIT frame (`msgType=3`) to mark the end of the input segment.

### Multi-Client Concurrency (Same Execution)

Expand Down
4 changes: 3 additions & 1 deletion docs/spear-console-overview-zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,9 @@ SPEAR Console 复用当前 execution stream 协议:

1) 在 “Start a chat” 弹窗中选择 Task -> Instance -> Execution。
2) Console 以同域方式调用 `POST /api/v1/executions/{execution_id}/streams/session` 获取 `ws_url`。
3) Console 通过 WebSocket 连接 `ws_url`,并使用 SSF frame 交互(stream id `1`,文本帧 `msgType=2`)。
3) Console 通过 WebSocket 连接 `ws_url`,并使用 SSF frame 交互。
- 连接建立(`open`)后,Console 会先发送一个初始 SSF v1 CTRL 帧(`msgType=1`,空 data),用于立刻建立 `stream_id=1`,避免等待用户首次输入。
- 用户输入会先发送一条或多条 DATA 帧(`msgType=2`),并在输入完成后发送 COMMIT 帧(`msgType=3`)标记该段输入结束。

### 多客户端并发(同一 execution)

Expand Down
4 changes: 2 additions & 2 deletions examples/kv_factory_usage.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
// KV存储工厂模式使用示例

use spear_next::storage::{
create_kv_store_from_config, create_kv_store_from_env, get_kv_store_factory,
set_kv_store_factory, DefaultKvStoreFactory, KvStore, KvStoreConfig, KvStoreFactory,
create_kv_store_from_config, create_kv_store_from_env, DefaultKvStoreFactory, KvStore,
KvStoreConfig, KvStoreFactory,
};
use std::env;

Expand Down
Loading
Loading