Skip to content

feat(server): add --api-key bearer authentication - #305

Open
benwilson wants to merge 1 commit into
FlashML-org:mainfrom
benwilson:feat/api-key
Open

feat(server): add --api-key bearer authentication#305
benwilson wants to merge 1 commit into
FlashML-org:mainfrom
benwilson:feat/api-key

Conversation

@benwilson

Copy link
Copy Markdown

Summary

  • add an optional, vLLM-compatible --api-key to ft serve (also read from FREETOKEN_API_KEY when the flag is absent, so the key need not appear in ps)
  • with a key set, every route except /health requires Authorization: Bearer <key>; anything else is answered 401 with a WWW-Authenticate: Bearer challenge and an OpenAI-shaped error body
  • unset, the server behaves exactly as today
  • shell mode keeps working: the attached client is handed the server's key, and ft shell attaching to a running server reads FREETOKEN_API_KEY
  • one row in docs/cli.md

Closes #152.

Motivation

ft serve currently has no way to require credentials, so any deployment that publishes the port beyond loopback is open, and clients that insist on a key (the report in #152) cannot be pointed at it without a reverse proxy in front. This is the minimal native path: one flag, one middleware, default unchanged — the same shape as #230 takes for TLS.

Design notes

  • What is guarded: everything but /health. The route surface includes mutating control routes (/v1/admin/prepare-stop, /v1/cache/rebuild), request history (/v1/requests) and /generate outside /v1, so a /v1-prefix rule would leave holes. /health stays open because liveness probes (load balancers, Docker healthchecks) and the desktop app's load-progress polling cannot carry a header and it reveals only status.
  • Ordering: the middleware is registered after _record_request_middleware, so Starlette runs it first — a 401 never lands in the request ring or a handler. OPTIONS passes through (a CORS preflight carries no credentials); install_cors runs at startup and stays outermost, so it answers the preflight and decorates the 401 with CORS headers.
  • Compare: hmac.compare_digest; the scheme is case-insensitive, whitespace around the token is not part of it.
  • Out of scope: the daemon's /engine/* proxy does not yet forward a key to a serve it manages (it cannot launch one with --api-key today), so nothing changes there.

Testing

  • PYTHONPATH=python python -m pytest tests/server -q563 passed (535 before + 28 in tests/server/test_api_key.py), on an RTX 3090 box with the [accel] install at this base
  • new tests cover: flag / env / precedence / empty-flag rejection / shell-mode parsing; 401 for a missing, wrong, truncated, over-long, Basic or bare credential; 200 for the matching bearer (case-insensitive scheme); /health open; every other route (/v1/chat/completions, /v1/messages, /v1/responses, /v1/models, /v1/stats, /v1/requests, /v1/cache/rebuild, /v1/admin/prepare-stop, /generate) gated before its handler; OPTIONS not challenged; install_api_key arm/disarm; the shell client sending the bearer on its control-plane requests
  • ft serve --help lists the flag next to --host/--port

I checked open and closed PRs for an existing api-key / auth implementation and found none (#152 is the open request; #230 is the adjacent TLS flag).

Require `Authorization: Bearer <key>` on every route except /health when
`ft serve --api-key` (or FREETOKEN_API_KEY) is set; 401 with a
WWW-Authenticate challenge otherwise. Unset, nothing changes.

- The check is a middleware registered after the request-ring middleware,
  so it runs first: a rejected request never reaches a handler or the
  ring. OPTIONS passes through for CORS preflights; the CORS middleware
  installed at startup stays outermost. Constant-time compare.
- /health stays open: liveness probes and the desktop app's load-progress
  polling cannot carry a header and reveal only status.
- Shell mode keeps working: the attached client is handed the server's
  key; `ft shell` attaching to a running server reads FREETOKEN_API_KEY.
- The control-plane requests of the shell client now send the same bearer
  the OpenAI client already sends.

Closes FlashML-org#152
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

API access security verification

1 participant