From 3b81c406dcc23b3ed76aa227bab49fe49ba04811 Mon Sep 17 00:00:00 2001 From: Eric Wang Date: Fri, 7 Aug 2026 05:14:06 -0700 Subject: [PATCH 1/3] feat(gateway): carry web_search, chart days instead of seconds, report upstream MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three changes that each started as a measurement. **web_search works on the free tier.** The old refusal argued that a server-side tool "performs billed work that never appears in the usage object". Measured against the live API: one search request made eleven server-side calls and billed 40,260 input tokens (32k cached), and the account balance moved by nothing beyond those tokens — eleven searches at a frontier vendor's $10/1k rate would have been 0.79 CNY and unmistakable. There is no per-search fee; the whole cost arrives as input tokens the meter already reads exactly. What the measurement did break is the reservation. `meter.Estimate` bounded input at one token per body byte, which is true for every request except this one — DeepSeek chooses how many pages to read. So a search reserves a 256k input allowance (~6x the observed case) and spends one of three daily searches, a new quota dimension because one search costs about what ten ordinary turns cost and the request counter alone would let one caller take a quarter of the day's budget while looking ordinary. Every other server-side tool stays refused. Within the allowance the budget is still a hard ceiling; the overshoot past it is bounded and stated in DESIGN.md. **The dashboard charts a day per bar.** It charted output tokens/sec over five minutes, which is the wrong instrument for a pool serving a few requests an hour: flat at zero whenever anyone looked. The 30-day series comes from `quota.History`, reading the same journals the money settles from, so the chart cannot disagree with the ledger. Quiet days are zeroes, not gaps. The live rates are still on the page, demoted to where their usual zero is honest rather than alarming. **"Is it us or DeepSeek?"** Every upstream round trip is recorded and `/v1/status` carries the last success, the fault streak and the last fault shape — first-party, because `status.deepseek.com` turns out to be both unparseable and unreachable from our host (Aliyun Beijing; TCP connects, TLS never completes, while api.deepseek.com answers fine from the same box). A status page also cannot see a route broken only from here. Probe results and the reasoning are in TASTE.md. Also: the economics page now cites measured cost-per-task from Artificial Analysis and SWE-rebench instead of only rate-card arithmetic — including the two figures that cut against the argument, since a page that hid them would be propaganda. Co-Authored-By: Claude Fable 5 --- AGENTS.md | 13 +- TASTE.md | 65 ++++++ gateway/DESIGN.md | 34 +++ gateway/cmd/dsgate/main.go | 2 + gateway/internal/meter/meter.go | 30 ++- gateway/internal/meter/meter_test.go | 22 +- gateway/internal/policy/policy.go | 51 ++++- gateway/internal/policy/policy_test.go | 33 ++- gateway/internal/quota/lifetime_test.go | 2 +- gateway/internal/quota/quota.go | 150 ++++++++++++- gateway/internal/quota/quota_test.go | 118 +++++++---- gateway/internal/server/proxy.go | 35 +++- gateway/internal/server/server_test.go | 53 +++++ gateway/internal/server/status.go | 17 ++ gateway/internal/server/status_test.go | 105 +++++++++- gateway/internal/server/web/app.js | 197 ++++++++++++------ gateway/internal/server/web/index.html | 58 +++++- .../internal/server/web/pages/economics.html | 45 +++- gateway/internal/server/web/style.css | 45 +++- gateway/internal/stats/stats.go | 104 ++++++++- 20 files changed, 1027 insertions(+), 152 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 781eb7b..7ff5c3d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -155,9 +155,16 @@ deepseek free off # forget the enrolment on this machine ``` Free-tier limits, per UTC day: 30 requests, 60K input tokens, 20K output -tokens, 4K output per call, 128KB per request body, `deepseek-v4-flash` -only. A request for pro is **refused, not downgraded**. `models` and -`status` cost no quota; everything that can generate a token does. +tokens, 3 web searches, 4K output per call, 128KB per request body, +`deepseek-v4-flash` only. A request for pro is **refused, not +downgraded**. `models` and `status` cost no quota; everything that can +generate a token does. + +`respond --web-search` works on the free tier and spends one of the three +daily searches. It is rationed that tightly because DeepSeek reads whole +pages into the prompt — one measured search request billed 40K input +tokens, about ten ordinary turns — so treat it as a few lookups a day, not +a research loop. Other server-side tools are still refused. Errors from the gateway carry `"type":"free_tier_*"` and a message that already contains the next step — do not append DeepSeek's own advice to diff --git a/TASTE.md b/TASTE.md index df428dc..5d53d4c 100644 --- a/TASTE.md +++ b/TASTE.md @@ -141,6 +141,24 @@ a human always has somewhere to go. **Expires.** If DeepSeek publishes a documented status JSON endpoint, consume it and report both. +**2026-08-07 — expiry tested, still holds, and now for a second reason.** +Probed for a JSON endpoint from the production host: `/api/v2/summary.json`, +`/api/v2/status.json`, `/api/status`, `/api/v1/status` and the bare page all +fail. `status.deepseek.com` resolves to an Aliyun Beijing load balancer +(`statuspage.flashcat.cloud`) that accepts the TCP connection and then never +completes the TLS handshake from outside China, while `api.deepseek.com` +answers fine from the same box. So the page is both unparseable and +unreachable from where our gateway runs. Consuming it would mean shipping a +health signal that is permanently unknown — worse than the wrong all-clear +this scar was written about, because it would also be *our* dashboard showing +red for *their* geography. What the gateway publishes instead is a +first-party observation: every round trip to `api.deepseek.com` is recorded, +and `/v1/status` carries an `upstream` block with the last success, the +consecutive-failure streak and the last fault shape. That answers the +question a visitor actually has — is this you or them — with data a status +page structurally cannot have, since it cannot see a route that is broken +only from here. The incident page stays linked for humans. + --- ## 2026-08-05 rejected: shipping the tokenizer, or estimating from the ratios @@ -345,3 +363,50 @@ vectors by coincidence and fail against the gateway every time. **Expires.** If the CLI and the gateway ever merge into one module, the Go halves should share code and only the browser stays separate. + +--- + +## 2026-08-07 accepted, after a measurement: `web_search` on the free tier + +Recorded here because it reverses an earlier refusal, and a reversal +without its evidence is just a mood swing. + +**The refusal it replaces.** `policy.forbidServerTools` rejected every +server-side tool with a real argument: such a tool "performs billed work +that never appears in the usage object, which would put its cost outside +every ceiling this gateway enforces". Correct reasoning, untested premise. + +**What the measurement showed.** One `respond --web-search` call against +the live API on 2026-08-07 made eleven server-side calls (searches, page +opens, an in-page find) and reported 40,260 input tokens, 32,000 of them +cache hits, 3,100 output. The account balance moved by nothing beyond +those tokens — eleven searches at a frontier vendor's $10-per-1,000 rate +would have been $0.11, or 0.79 CNY, and unmistakable against a 14.26 CNY +balance. So the premise was wrong in the half that mattered: there is no +per-search fee, and the entire cost of a search arrives as input tokens +that this gateway already meters exactly. + +**What the measurement did break.** The half of the premise that was +right, in a different place than expected. `meter.Estimate` bounded a +request's input at one token per body byte — true for every other request +and false for this one, because DeepSeek chooses how many pages to read. +A search request's input is upstream-controlled, so the admission +reservation no longer bounds it from the body. + +**Reuse.** Allow `web_search` (both the bare and dated tool names), refuse +every other server-side tool still, and pay for the change in two places: +a 256k-token input allowance in the reservation, roughly six times the +observed case; and a per-user daily ration of three searches, which is a +new quota dimension because one search costs about what ten ordinary turns +cost and the request counter alone would let one caller quietly take a +quarter of the day's budget. Stating the trade honestly: within the +allowance the budget is still a hard ceiling, and beyond it a search can +overshoot by the difference, bounded by the few distinct callers who can +be mid-search at once given a per-subject in-flight cap of one. + +**Expires.** If DeepSeek starts billing searches separately, or documents +a cap on injected search context, redo the arithmetic — the allowance and +the ration are both sized to a single measurement and should be re-measured +when the tool changes. If a search request is ever observed above 256k +input tokens in production, that is the signal to raise the allowance +rather than to quietly accept the overshoot. diff --git a/gateway/DESIGN.md b/gateway/DESIGN.md index 7f0207f..73725ae 100644 --- a/gateway/DESIGN.md +++ b/gateway/DESIGN.md @@ -300,6 +300,7 @@ dollar.** | `DSGATE_ANON_DAILY_REQUESTS` | 30 | enough to be useful for a day's work, not enough to script against | | `DSGATE_ANON_DAILY_INPUT_TOKENS` | 60000 | ~150 pages of context per day | | `DSGATE_ANON_DAILY_OUTPUT_TOKENS` | 20000 | the expensive side; the real cap | +| `DSGATE_ANON_DAILY_SEARCHES` | 3 | a `web_search` request costs ~10 ordinary turns; the request counter alone would let one caller take a quarter of the day | | `DSGATE_ANON_MAX_TOKENS` | 4096 | clamps a single response, bounding overshoot | | `DSGATE_MAX_BODY_BYTES` | 131072 | ~32K tokens; bounds the input side of overshoot | | `DSGATE_DAILY_BUDGET_USD` | 1.00 | the circuit breaker; the number that actually protects us | @@ -312,6 +313,19 @@ dollar.** | `DSGATE_TOKEN_TTL_DAYS` | 7 | identities age out instead of accumulating | | `DSGATE_BALANCE_CHECK_MINUTES` | 15 | the ledger's "we have credit" is checked against the real account | +**`web_search` is carried, and rationed.** Measured on 2026-08-07, one +search request made eleven server-side calls and billed 40,260 input +tokens with no separate per-search fee — so its whole cost arrives as +input tokens the meter already reads. What it breaks is the *reservation*, +which bounded input at one token per body byte: DeepSeek chooses how many +pages to read, so a search's input is upstream-controlled. Hence a 256k +input allowance at admission (about 6x the observed case) plus the daily +ration above. Within that allowance the budget is still a hard ceiling; +past it a search can overshoot by the difference, bounded by how many +distinct callers can be mid-search at once. Every other server-side tool +stays refused: unknown work at an unknown price, spent from donated +credit. Reasoning and the expiry condition are in `TASTE.md`. + **Free tier is flash only.** Pro is 3x the price and the request is *rejected*, not silently downgraded — a user who asked for pro and got flash without being told would draw wrong conclusions and blame the model. @@ -333,6 +347,8 @@ What that document says and what it refuses to say are both deliberate: | subject ids **truncated to 6 chars** | whole subject ids | a whole id can be matched against the one in someone's `free.json` | | **per-country** request counts | anything per-IP, ever | an aggregate is a fact about the service, not about a person | | token counts, tok/s, live subjects | prompts, completions | the promise in `deepseek free` is the promise here | +| **requests per day** for 30 days | per-day spend | a spend series is a map of when we are cheapest to empty | +| **upstream** last-success, fault streak | upstream key or account detail | "is it you or DeepSeek" is the question a failing caller has | Exact money, per-key health and the full subject table live behind `GET /admin/status` with the operator token. @@ -342,6 +358,24 @@ maps, everything in memory and lost on restart. That is the right trade — losing it costs a graph, and it keeps observability from ever becoming a second, weaker copy of the money. +**The main chart is a day per bar, not a second.** It was output tokens +per second over five minutes, which is the wrong instrument for a pool +serving a few requests an hour: idle almost every second, so the line was +flat at zero whenever anyone looked and said nothing true about whether +the service works. The daily series comes from `quota.History`, which +reads the journals — the same files the money settles from, so the pretty +chart cannot disagree with the ledger. Finished days are immutable and +memoised; only today is recomputed. Quiet days are present as zeroes, +because a gap in a chart reads as missing data rather than as a quiet day. + +**Upstream health is first-party, not scraped.** Every round trip to +DeepSeek is recorded with its outcome, and only *their* failures (429, 5xx, +transport) count as faults — a caller's own 4xx would otherwise read as an +outage. Three consecutive faults reads as `down`, one as `degraded`, and +before the first forwarded request the state is `unknown` rather than a +cheerful all-clear it has not earned. `status.deepseek.com` is deliberately +not consumed; see `TASTE.md` for the probe results. + ## The key pool One key was a single point of failure with a hard floor. Package diff --git a/gateway/cmd/dsgate/main.go b/gateway/cmd/dsgate/main.go index 4656e32..7cb559b 100644 --- a/gateway/cmd/dsgate/main.go +++ b/gateway/cmd/dsgate/main.go @@ -74,6 +74,7 @@ Per-user daily limits: DSGATE_ANON_DAILY_REQUESTS (30) DSGATE_ANON_DAILY_INPUT_TOKENS (60000) DSGATE_ANON_DAILY_OUTPUT_TOKENS (20000) + DSGATE_ANON_DAILY_SEARCHES (3) server-side web searches per user DSGATE_ANON_MAX_TOKENS (4096) per-request output cap DSGATE_MAX_BODY_BYTES (131072) per-request body cap DSGATE_REQUESTS_PER_MINUTE (20) per-address burst @@ -132,6 +133,7 @@ func run() error { DailyRequests: envInt("DSGATE_ANON_DAILY_REQUESTS", 30), DailyInputTokens: envInt("DSGATE_ANON_DAILY_INPUT_TOKENS", 60000), DailyOutputTokens: envInt("DSGATE_ANON_DAILY_OUTPUT_TOKENS", 20000), + DailySearches: envInt("DSGATE_ANON_DAILY_SEARCHES", 3), DailyBudgetUSD: envFloat("DSGATE_DAILY_BUDGET_USD", 1.00), TotalBudgetUSD: envFloat("DSGATE_TOTAL_BUDGET_USD", 20.00), } diff --git a/gateway/internal/meter/meter.go b/gateway/internal/meter/meter.go index 02aa17e..8cd7393 100644 --- a/gateway/internal/meter/meter.go +++ b/gateway/internal/meter/meter.go @@ -89,9 +89,17 @@ func Cost(model string, u Usage) float64 { // default upstream, and DeepSeek does not document that reasoning // tokens respect max_tokens — they are billed as output either way, // so the bound assumes they do not. -func Estimate(model string, requestBytes, maxTokens int) float64 { +// +// A search request breaks the first rule: the pages DeepSeek reads on the +// caller's behalf arrive as input tokens the body never contained, so +// searchInputAllowance is added to the input bound instead. +func Estimate(model string, requestBytes, maxTokens int, search bool) float64 { + input := requestBytes + 1 + if search { + input += searchInputAllowance + } return Cost(model, Usage{ - InputTokens: requestBytes + 1, + InputTokens: input, OutputTokens: maxTokens + reasoningAllowance, Found: false, }) @@ -103,6 +111,24 @@ func Estimate(model string, requestBytes, maxTokens int) float64 { // a cent, so over-reserving costs headroom, not money. const reasoningAllowance = 32 << 10 +// searchInputAllowance is the input headroom reserved for a server-side +// web search, whose page reads land in input_tokens without ever passing +// through the request body. +// +// 256k is a judgement, not a proof. A search request measured live on +// 2026-08-07 reported 40,260 input tokens after eleven server-side calls, +// so this is roughly six times the observed case; the model's 1M context +// is the only true bound, and reserving 1M would price a single search at +// more than half a day's budget and make the feature unofferable. +// +// The honest statement of the trade: within this allowance the budget is +// still a hard ceiling, and beyond it a search request can overshoot by +// the difference. Two things keep that survivable — the per-subject +// in-flight cap means one caller cannot stack such requests, and searches +// are rationed per user per day, so the overshoot is bounded by the few +// distinct callers who can be mid-search at the same moment. +const searchInputAllowance = 256 << 10 + // rawUsage is permissive on purpose: it decodes the usage object of every // format at once, using pointers so "absent" and "zero" stay distinct. // Which fields are present is what identifies the format. diff --git a/gateway/internal/meter/meter_test.go b/gateway/internal/meter/meter_test.go index f5f6bef..f778425 100644 --- a/gateway/internal/meter/meter_test.go +++ b/gateway/internal/meter/meter_test.go @@ -176,7 +176,7 @@ func TestEstimateExceedsATypicalRealCharge(t *testing.T) { const body = 4000 const maxTokens = 4096 - est := Estimate("deepseek-v4-flash", body, maxTokens) + est := Estimate("deepseek-v4-flash", body, maxTokens, false) real := Cost("deepseek-v4-flash", Usage{InputTokens: body / 3, OutputTokens: 800, Found: true}) if est <= real { t.Errorf("estimate %v is not above a realistic charge %v; unbillable would be cheaper than billable", est, real) @@ -218,3 +218,23 @@ func itoa(n int) string { } return string(b) } + +// A search request's input is chosen by DeepSeek, not by the caller: the +// pages it reads are billed as input tokens that never passed through the +// body. So the reservation cannot be derived from the body alone, and a +// search must hold materially more than the same bytes without one. +func TestSearchReservesBeyondTheBody(t *testing.T) { + const model = "deepseek-v4-flash" + plain := Estimate(model, 400, 1000, false) + search := Estimate(model, 400, 1000, true) + + if search <= plain { + t.Fatalf("a search reserved %v, no more than the %v an ordinary request holds", search, plain) + } + // The measured case on 2026-08-07 was 40k input tokens; the reservation + // has to cover that with room, or the ceiling leaks on every search. + measured := Cost(model, Usage{InputTokens: 40_260, CacheHitTokens: 32_000, OutputTokens: 3_100}) + if search < measured { + t.Errorf("reservation %v is under the %v a real measured search cost", search, measured) + } +} diff --git a/gateway/internal/policy/policy.go b/gateway/internal/policy/policy.go index 67484ab..ffaedc1 100644 --- a/gateway/internal/policy/policy.go +++ b/gateway/internal/policy/policy.go @@ -88,6 +88,12 @@ type Decision struct { // estimate if the response turns out to be unmeterable. MaxTokens int Stream bool + // Search is set when the request asks for DeepSeek's server-side web + // search. It travels because such a request costs a multiple of an + // ordinary one: the server injects the pages it read as input tokens, + // so neither the body's size nor MaxTokens predicts the bill. The + // reservation and the per-user ration both key off this. + Search bool } // Reject is a request refused before it cost anything. @@ -131,7 +137,7 @@ func Apply(route Route, body []byte, subject string, lim Limits) (*Decision, err if err := forbidFanOut(obj); err != nil { return nil, err } - if err := forbidServerTools(obj, route.Format); err != nil { + if err := checkServerTools(obj, route.Format, d); err != nil { return nil, err } setIdentity(obj, route.Format, subject) @@ -233,12 +239,29 @@ func forbidFanOut(obj map[string]any) error { return nil } -// forbidServerTools refuses tools that run on DeepSeek's side. Client -// tools ("function") only declare a schema and cost nothing extra; a -// server-side tool like web_search performs billed work that never -// appears in the usage object, which would put its cost outside every -// ceiling this gateway enforces. Only the Responses format offers them. -func forbidServerTools(obj map[string]any, f Format) error { +// checkServerTools decides which tools that run on DeepSeek's side the +// free tier will carry. Client tools ("function") only declare a schema +// and cost nothing extra. Only the Responses format offers server-side +// ones at all. +// +// web_search is allowed, and the reason is a measurement rather than a +// guess. Against the live API on 2026-08-07, one search request made 11 +// server-side calls (searches, page opens, an in-page find) and reported +// 40,260 input tokens, 32,000 of them cache hits — and the account +// balance moved by nothing beyond those tokens. So DeepSeek charges no +// per-search fee: the whole cost of a search arrives as input tokens in +// the usage object, which is exactly what this gateway already meters. +// Eleven searches at a frontier vendor's $10-per-1,000 rate would have +// been $0.11 and unmistakable in the balance; it was not there. +// +// What that measurement does change is the reservation. A search +// request's input is chosen by the server, not by the caller, so the +// request body no longer bounds it — see meter.Estimate. +// +// Every other server-side tool stays refused: an unknown tool is unknown +// work at an unknown price, and the honest default for spending someone +// else's donated credit is no. +func checkServerTools(obj map[string]any, f Format, d *Decision) error { if f != FormatResponses { return nil } @@ -246,16 +269,26 @@ func forbidServerTools(obj map[string]any, f Format) error { for _, t := range tools { tool, _ := t.(map[string]any) kind, _ := tool["type"].(string) - if kind != "" && kind != "function" { + switch { + case kind == "" || kind == "function": + case isWebSearch(kind): + d.Search = true + default: return &Reject{ Message: fmt.Sprintf("the free tier does not serve server-side tools (%q)", kind), - Hint: "bring your own key for web search: https://platform.deepseek.com/api_keys", + Hint: "web_search works here; for anything else bring your own key: https://platform.deepseek.com/api_keys", } } } return nil } +// isWebSearch matches the tool DeepSeek documents under two names, the +// bare one and the dated one their Responses API also accepts. +func isWebSearch(kind string) bool { + return kind == "web_search" || strings.HasPrefix(kind, "web_search_") +} + // setIdentity stamps the subject onto the request in whichever field the // format actually reads. // diff --git a/gateway/internal/policy/policy_test.go b/gateway/internal/policy/policy_test.go index 1e125c5..8671657 100644 --- a/gateway/internal/policy/policy_test.go +++ b/gateway/internal/policy/policy_test.go @@ -275,20 +275,39 @@ func asReject(err error, target **Reject) bool { return ok } -// Server-side tools perform billed work that never appears in the usage -// object — outside every ceiling this gateway enforces. Client function -// tools only declare a schema and stay allowed. +// web_search is carried, because measurement showed its whole cost +// arrives as input tokens this gateway already meters. Every other +// server-side tool is unknown work at an unknown price and stays refused. +// Client function tools only declare a schema and were never in question. func TestServerSideToolsAreRefused(t *testing.T) { route, _ := Lookup("POST", "/responses") lim := Limits{MaxTokens: 100, Model: "deepseek-v4-flash"} - _, err := Apply(route, []byte(`{"input":"hi","tools":[{"type":"web_search"}]}`), "sub", lim) - if err == nil { - t.Fatal("web_search passed policy; its per-search cost has no ceiling") + for _, kind := range []string{"web_search", "web_search_2025_08_26"} { + d, err := Apply(route, []byte(`{"input":"hi","tools":[{"type":"`+kind+`"}]}`), "sub", lim) + if err != nil { + t.Fatalf("%s was refused: %v", kind, err) + } + if !d.Search { + t.Errorf("%s did not set Decision.Search, so it would be reserved and rationed as an ordinary request", kind) + } + } + + // An unknown server-side tool is still a refusal, and the message has + // to point at the one that does work rather than only at the exit. + var rej *Reject + _, err := Apply(route, []byte(`{"input":"hi","tools":[{"type":"code_interpreter"}]}`), "sub", lim) + if !asReject(err, &rej) { + t.Fatalf("an unknown server-side tool passed policy: %v", err) + } + if !strings.Contains(rej.Hint, "web_search") { + t.Errorf("the refusal does not mention the tool that works: %q", rej.Hint) } - if _, err := Apply(route, []byte(`{"input":"hi","tools":[{"type":"function","name":"f"}]}`), "sub", lim); err != nil { + if d, err := Apply(route, []byte(`{"input":"hi","tools":[{"type":"function","name":"f"}]}`), "sub", lim); err != nil { t.Errorf("a client function tool was refused: %v", err) + } else if d.Search { + t.Error("a function tool was counted as a search") } // The other formats have no server-side tools; their tools stay open. diff --git a/gateway/internal/quota/lifetime_test.go b/gateway/internal/quota/lifetime_test.go index 9c8494c..c0db0f4 100644 --- a/gateway/internal/quota/lifetime_test.go +++ b/gateway/internal/quota/lifetime_test.go @@ -171,7 +171,7 @@ func TestCreditPoolStaysExhaustedAcrossARestart(t *testing.T) { l, done := open(t, dir, poolLimits()) defer done() - err := l.Admit("a-brand-new-subject", 0) + err := l.Admit("a-brand-new-subject", Admission{}) if err == nil { t.Fatal("admitted a request with the credit pool already empty") } diff --git a/gateway/internal/quota/quota.go b/gateway/internal/quota/quota.go index dfe41e3..bff8c6d 100644 --- a/gateway/internal/quota/quota.go +++ b/gateway/internal/quota/quota.go @@ -31,6 +31,7 @@ type Limits struct { DailyRequests int DailyInputTokens int DailyOutputTokens int + DailySearches int // DailyBudgetUSD is the circuit breaker: the total this service may // spend across all users in one UTC day. This is the number that @@ -48,6 +49,7 @@ type Account struct { Requests int `json:"requests"` InputTokens int `json:"input_tokens"` OutputTokens int `json:"output_tokens"` + Searches int `json:"searches"` SpentUSD float64 `json:"spent_usd"` } @@ -71,6 +73,22 @@ type UserCaps struct { Requests int `json:"requests"` InputTokens int `json:"input_tokens"` OutputTokens int `json:"output_tokens"` + // Searches rations requests that use DeepSeek's server-side web + // search. It exists because such a request costs roughly ten times an + // ordinary turn — the pages it reads are billed as input tokens — so + // the request count alone would let one caller take a large share of + // the day's budget while looking like a normal user. + Searches int `json:"searches"` +} + +// Admission is what a request asks the ledger for before it is forwarded. +// It is a struct rather than another positional argument because the two +// fields answer different questions — how much money to hold, and which +// per-user ration to spend — and a bare `true` at a call site would say +// neither. +type Admission struct { + ReserveUSD float64 + Search bool } // Reason classifies a refusal so the HTTP layer can pick a status code @@ -81,6 +99,7 @@ const ( ReasonRequests Reason = "daily_requests" ReasonInputTokens Reason = "daily_input_tokens" ReasonOutputTokens Reason = "daily_output_tokens" + ReasonSearches Reason = "daily_searches" ReasonDailyBudget Reason = "daily_budget" ReasonCredits Reason = "credits_exhausted" ReasonRevoked Reason = "revoked" @@ -109,6 +128,14 @@ func (e *LimitError) Error() string { return "this token has been revoked" case ReasonUnavailable: return "the free tier cannot record spend right now" + case ReasonRequests: + return "you have used today's request allowance" + case ReasonInputTokens: + return "you have used today's input-token allowance" + case ReasonOutputTokens: + return "you have used today's output-token allowance" + case ReasonSearches: + return "you have used today's web-search allowance" default: return fmt.Sprintf("daily %s limit reached", string(e.Reason)) } @@ -162,6 +189,12 @@ type Ledger struct { journalErr error now func() time.Time + + // history memoises finished days for the dashboard's daily series. It + // has its own lock because it is read on a status request and must not + // queue behind a request being admitted. + histMu sync.Mutex + history map[string]Day } // entry is one line of the journal. @@ -492,7 +525,8 @@ func (l *Ledger) reopenLocked() { // admitted only if, priced at its absolute maximum, it still fits under // both. Charge and Refund release the reservation, so the actual (almost // always much smaller) cost is what sticks. -func (l *Ledger) Admit(subject string, reserveUSD float64) error { +func (l *Ledger) Admit(subject string, req Admission) error { + reserveUSD := req.ReserveUSD l.mu.Lock() defer l.mu.Unlock() l.rollLocked() @@ -530,9 +564,17 @@ func (l *Ledger) Admit(subject string, reserveUSD float64) error { return &LimitError{Reason: ReasonInputTokens, ResetsAt: reset} case a.OutputTokens >= l.limits.DailyOutputTokens: return &LimitError{Reason: ReasonOutputTokens, ResetsAt: reset} + case req.Search && a.Searches >= l.limits.DailySearches: + return &LimitError{Reason: ReasonSearches, ResetsAt: reset} } a.Requests++ + if req.Search { + // Counted at admission rather than at settlement, because the + // ration has to bind before the money is spent: a search that + // failed still cost us the pages DeepSeek read. + a.Searches++ + } l.reserved += reserveUSD return nil } @@ -543,13 +585,18 @@ func (l *Ledger) Admit(subject string, reserveUSD float64) error { // Only failures the caller cannot provoke qualify — transport errors and // upstream 429/5xx. Refunding on anything the client controls, such as a // malformed body, would turn the request counter into a free retry loop. -func (l *Ledger) Refund(subject string, reserveUSD float64) { +func (l *Ledger) Refund(subject string, req Admission) { l.mu.Lock() defer l.mu.Unlock() - if a, ok := l.accounts[subject]; ok && a.Requests > 0 { - a.Requests-- + if a, ok := l.accounts[subject]; ok { + if a.Requests > 0 { + a.Requests-- + } + if req.Search && a.Searches > 0 { + a.Searches-- + } } - l.releaseLocked(reserveUSD) + l.releaseLocked(req.ReserveUSD) } // Release gives back a reservation while keeping the request debit, for @@ -629,6 +676,7 @@ func (l *Ledger) Status(subject, tier string) Status { Requests: l.limits.DailyRequests, InputTokens: l.limits.DailyInputTokens, OutputTokens: l.limits.DailyOutputTokens, + Searches: l.limits.DailySearches, }, ResetsAt: midnight(l.now()), Exhausted: l.priorSpend+l.daySpend >= l.limits.TotalBudgetUSD, @@ -794,3 +842,95 @@ func midnight(t time.Time) time.Time { u := t.UTC() return time.Date(u.Year(), u.Month(), u.Day(), 0, 0, 0, 0, time.UTC).Add(24 * time.Hour) } + +// Day is one UTC day of traffic, for the dashboard's history. +// +// No money, deliberately, for the same reason the public status document +// withholds it: a per-day spend series is a map of how much it takes to +// empty this service and when it is cheapest to try. +type Day struct { + Date string `json:"date"` + Requests int `json:"requests"` + InputTokens int `json:"input_tokens"` + OutputTokens int `json:"output_tokens"` + // Subjects is how many distinct anonymous identities sent something + // that day — the closest honest thing to "people", since no account + // exists. Only the count is published, never the ids. + Subjects int `json:"subjects"` +} + +// History is the last n UTC days, oldest first, including today. +// +// It is read from the journals rather than from a new counter, because the +// journals are already the record the money is settled from — a separate +// series could disagree with the ledger, and then the pretty chart would +// be the one people believe. A day that never had traffic is present with +// zeroes, so the series is a calendar rather than a list of events and the +// gaps are visible. +// +// Past days are immutable once the date rolls, so each is scanned at most +// once per process; only today is recomputed, from the live counters. +func (l *Ledger) History(days int) []Day { + if days < 1 { + days = 1 + } + l.mu.Lock() + l.rollLocked() + today := l.day + live := Day{Date: today, Subjects: len(l.accounts)} + for _, a := range l.accounts { + live.Requests += a.Requests + live.InputTokens += a.InputTokens + live.OutputTokens += a.OutputTokens + } + end, err := time.Parse("2006-01-02", today) + l.mu.Unlock() + if err != nil { + return []Day{live} + } + + out := make([]Day, 0, days) + for i := days - 1; i >= 0; i-- { + date := end.AddDate(0, 0, -i).Format("2006-01-02") + if date == today { + out = append(out, live) + continue + } + out = append(out, l.pastDay(date)) + } + return out +} + +// pastDay reads one finished day out of its journal, remembering the +// answer: a day that has ended cannot change. +func (l *Ledger) pastDay(date string) Day { + l.histMu.Lock() + if d, ok := l.history[date]; ok { + l.histMu.Unlock() + return d + } + l.histMu.Unlock() + + d := Day{Date: date} + if f, err := os.Open(l.journalPath(date)); err == nil { + seen := make(map[string]struct{}) + scanJournal(f, func(e entry) { + d.Requests++ + d.InputTokens += e.InputTokens + d.OutputTokens += e.OutputTokens + if e.Subject != "" { + seen[e.Subject] = struct{}{} + } + }) + f.Close() + d.Subjects = len(seen) + } + + l.histMu.Lock() + if l.history == nil { + l.history = make(map[string]Day) + } + l.history[date] = d + l.histMu.Unlock() + return d +} diff --git a/gateway/internal/quota/quota_test.go b/gateway/internal/quota/quota_test.go index 1d3e368..68cafaf 100644 --- a/gateway/internal/quota/quota_test.go +++ b/gateway/internal/quota/quota_test.go @@ -42,11 +42,11 @@ func TestRequestsAreCappedPerDay(t *testing.T) { defer done() for i := 0; i < 3; i++ { - if err := l.Admit("alice", 0); err != nil { + if err := l.Admit("alice", Admission{}); err != nil { t.Fatalf("request %d refused: %v", i+1, err) } } - err := l.Admit("alice", 0) + err := l.Admit("alice", Admission{}) if err == nil { t.Fatal("a fourth request was admitted against a limit of three") } @@ -55,7 +55,7 @@ func TestRequestsAreCappedPerDay(t *testing.T) { } // One subject hitting its limit must not affect another. - if err := l.Admit("bob", 0); err != nil { + if err := l.Admit("bob", Admission{}); err != nil { t.Errorf("bob was refused because alice ran out: %v", err) } } @@ -66,12 +66,12 @@ func TestTokenLimitsAreEnforcedAfterCharging(t *testing.T) { l, done := open(t, t.TempDir(), lim) defer done() - if err := l.Admit("alice", 0); err != nil { + if err := l.Admit("alice", Admission{}); err != nil { t.Fatal(err) } l.Charge("alice", "chat", "deepseek-v4-flash", 0, 0, 600, 0.0001, 0, false) - err := l.Admit("alice", 0) + err := l.Admit("alice", Admission{}) if err == nil { t.Fatal("admitted after the output token cap was passed") } @@ -94,7 +94,7 @@ func TestDailyBudgetStopsEveryone(t *testing.T) { spenders := []string{"a", "b", "c", "d", "e"} tripped := "" for _, who := range spenders { - if err := l.Admit(who, 0); err != nil { + if err := l.Admit(who, Admission{}); err != nil { tripped = who break } @@ -106,7 +106,7 @@ func TestDailyBudgetStopsEveryone(t *testing.T) { // And it holds for someone who has spent nothing at all: the breaker // is about the service's money, not the caller's behaviour. - err := l.Admit("someone-brand-new", 0) + err := l.Admit("someone-brand-new", Admission{}) if err == nil { t.Fatal("a fresh subject was admitted after the daily budget was spent") } @@ -122,10 +122,10 @@ func TestCreditsExhaustedOutranksTheDailyBudget(t *testing.T) { l, done := open(t, t.TempDir(), lim) defer done() - l.Admit("a", 0) + l.Admit("a", Admission{}) l.Charge("a", "chat", "deepseek-v4-flash", 0, 0, 0, 0.10, 0, false) - err := l.Admit("b", 0) + err := l.Admit("b", Admission{}) if got := reasonOf(t, err); got != ReasonCredits { t.Fatalf("reason = %q, want %q", got, ReasonCredits) } @@ -144,9 +144,9 @@ func TestCountersSurviveARestart(t *testing.T) { dir := t.TempDir() l, _ := open(t, dir, testLimits()) - l.Admit("alice", 0) + l.Admit("alice", Admission{}) l.Charge("alice", "chat", "deepseek-v4-flash", 400, 0, 200, 0.002, 0, false) - l.Admit("alice", 0) + l.Admit("alice", Admission{}) l.Charge("alice", "chat", "deepseek-v4-flash", 100, 0, 50, 0.001, 0, false) l.Close() @@ -170,7 +170,7 @@ func TestCountersSurviveARestart(t *testing.T) { func TestTruncatedJournalKeepsWhatItCan(t *testing.T) { dir := t.TempDir() l, _ := open(t, dir, testLimits()) - l.Admit("alice", 0) + l.Admit("alice", Admission{}) l.Charge("alice", "chat", "deepseek-v4-flash", 400, 0, 200, 0.002, 0, false) day := l.day l.Close() @@ -199,7 +199,7 @@ func TestLifetimeSpendSurvivesTheDayRolling(t *testing.T) { l, _ := open(t, dir, testLimits()) l.SetClock(func() time.Time { return now }) - l.Admit("alice", 0) + l.Admit("alice", Admission{}) l.Charge("alice", "chat", "deepseek-v4-flash", 100, 0, 100, 0.05, 0, false) now = now.Add(2 * time.Minute) // past midnight UTC @@ -241,14 +241,14 @@ func TestRefundReturnsTheRequestAllowance(t *testing.T) { l, done := open(t, t.TempDir(), testLimits()) defer done() - l.Admit("alice", 0) - l.Refund("alice", 0) + l.Admit("alice", Admission{}) + l.Refund("alice", Admission{}) if st := l.Status("alice", "anon"); st.Used.Requests != 0 { t.Errorf("requests after refund = %d, want 0", st.Used.Requests) } // Refunding more than was taken must not create allowance. - l.Refund("alice", 0) - l.Refund("alice", 0) + l.Refund("alice", Admission{}) + l.Refund("alice", Admission{}) if st := l.Status("alice", "anon"); st.Used.Requests != 0 { t.Errorf("over-refunding produced %d requests", st.Used.Requests) } @@ -259,7 +259,7 @@ func TestRevocation(t *testing.T) { l, done := open(t, dir, testLimits()) defer done() - if err := l.Admit("spammer", 0); err != nil { + if err := l.Admit("spammer", Admission{}); err != nil { t.Fatal(err) } if err := os.WriteFile(filepath.Join(dir, "revoked.txt"), @@ -270,10 +270,10 @@ func TestRevocation(t *testing.T) { t.Fatal(err) } - if got := reasonOf(t, l.Admit("spammer", 0)); got != ReasonRevoked { + if got := reasonOf(t, l.Admit("spammer", Admission{})); got != ReasonRevoked { t.Errorf("reason = %q, want %q", got, ReasonRevoked) } - if err := l.Admit("alice", 0); err != nil { + if err := l.Admit("alice", Admission{}); err != nil { t.Errorf("revoking one subject blocked another: %v", err) } } @@ -284,7 +284,7 @@ func TestRevocation(t *testing.T) { func TestJournalRecordsCountsAndNothingElse(t *testing.T) { dir := t.TempDir() l, _ := open(t, dir, testLimits()) - l.Admit("alice", 0) + l.Admit("alice", Admission{}) l.Charge("alice", "chat", "deepseek-v4-flash", 400, 120, 200, 0.002, 0, false) day := l.day l.Close() @@ -309,7 +309,7 @@ func TestJournalRecordsCountsAndNothingElse(t *testing.T) { func TestStatusDoesNotLeakServiceFinances(t *testing.T) { l, done := open(t, t.TempDir(), testLimits()) defer done() - l.Admit("alice", 0) + l.Admit("alice", Admission{}) l.Charge("alice", "chat", "deepseek-v4-flash", 10, 0, 10, 0.005, 0, false) st := l.Status("alice", "anon") @@ -333,20 +333,20 @@ func TestReservationIsACeiling(t *testing.T) { l, done := open(t, t.TempDir(), lim) defer done() - if got := reasonOf(t, l.Admit("a", 0.02)); got != ReasonDailyBudget { + if got := reasonOf(t, l.Admit("a", Admission{ReserveUSD: 0.02})); got != ReasonDailyBudget { t.Fatalf("a $0.02 worst case fit under a $0.01 budget: %v", got) } - if err := l.Admit("a", 0.005); err != nil { + if err := l.Admit("a", Admission{ReserveUSD: 0.005}); err != nil { t.Fatalf("a fitting reservation was refused: %v", err) } // A second request that fits the budget alone, but not alongside the // first one's reservation, must wait. - if got := reasonOf(t, l.Admit("b", 0.006)); got != ReasonDailyBudget { + if got := reasonOf(t, l.Admit("b", Admission{ReserveUSD: 0.006})); got != ReasonDailyBudget { t.Fatalf("overlapping reservations overshot the budget: %v", got) } // Settling the first request at its (small) real cost frees the room. l.Charge("a", "chat", "deepseek-v4-flash", 10, 0, 10, 0.001, 0.005, false) - if err := l.Admit("b", 0.006); err != nil { + if err := l.Admit("b", Admission{ReserveUSD: 0.006}); err != nil { t.Fatalf("room was not released at Charge: %v", err) } } @@ -357,15 +357,15 @@ func TestRefundReleasesTheReservation(t *testing.T) { l, done := open(t, t.TempDir(), lim) defer done() - if err := l.Admit("a", 0.009); err != nil { + if err := l.Admit("a", Admission{ReserveUSD: 0.009}); err != nil { t.Fatal(err) } - l.Refund("a", 0.009) - if err := l.Admit("b", 0.009); err != nil { + l.Refund("a", Admission{ReserveUSD: 0.009}) + if err := l.Admit("b", Admission{ReserveUSD: 0.009}); err != nil { t.Fatalf("a refunded reservation still held the budget: %v", err) } l.Release(0.009) - if err := l.Admit("c", 0.009); err != nil { + if err := l.Admit("c", Admission{ReserveUSD: 0.009}); err != nil { t.Fatalf("Release did not free the room: %v", err) } } @@ -376,7 +376,7 @@ func TestJournalFailureFailsClosed(t *testing.T) { l, done := open(t, t.TempDir(), testLimits()) defer done() - if err := l.Admit("a", 0); err != nil { + if err := l.Admit("a", Admission{}); err != nil { t.Fatal(err) } @@ -391,7 +391,7 @@ func TestJournalFailureFailsClosed(t *testing.T) { } l.Charge("a", "chat", "deepseek-v4-flash", 10, 0, 10, 0.0001, 0, false) - err := l.Admit("b", 0) + err := l.Admit("b", Admission{}) if got := reasonOf(t, err); got != ReasonUnavailable { t.Fatalf("admissions continued with a dead journal: %v", err) } @@ -400,7 +400,7 @@ func TestJournalFailureFailsClosed(t *testing.T) { if err := os.Chmod(path, 0o600); err != nil { t.Fatal(err) } - if err := l.Admit("b", 0); err != nil { + if err := l.Admit("b", Admission{}); err != nil { t.Fatalf("the ledger did not recover after the journal came back: %v", err) } } @@ -410,7 +410,7 @@ func TestJournalFailureFailsClosed(t *testing.T) { func TestReplaySkipsACorruptLine(t *testing.T) { dir := t.TempDir() l, done := open(t, dir, testLimits()) - l.Admit("a", 0) + l.Admit("a", Admission{}) l.Charge("a", "chat", "deepseek-v4-flash", 10, 0, 10, 0.002, 0, false) day := l.day done() @@ -424,7 +424,7 @@ func TestReplaySkipsACorruptLine(t *testing.T) { f.Close() l2, done2 := open(t, dir, testLimits()) - l2.Admit("a", 0) + l2.Admit("a", Admission{}) l2.Charge("a", "chat", "deepseek-v4-flash", 10, 0, 10, 0.003, 0, false) done2() @@ -434,3 +434,51 @@ func TestReplaySkipsACorruptLine(t *testing.T) { t.Errorf("day spend replayed as $%.4f; the corrupt line ate the entries after it", got) } } + +// A search costs roughly ten times an ordinary turn, so it has its own +// ration. Two properties matter and neither is obvious: running out of +// searches must not touch the rest of the tier, and a search that never +// reached the model must give the ration back. +func TestSearchesAreRationedWithoutBlockingOrdinaryRequests(t *testing.T) { + lim := testLimits() + lim.DailyRequests = 100 + lim.DailySearches = 2 + l, done := open(t, t.TempDir(), lim) + defer done() + + for i := 0; i < 2; i++ { + if err := l.Admit("alice", Admission{Search: true}); err != nil { + t.Fatalf("search %d refused: %v", i+1, err) + } + } + + err := l.Admit("alice", Admission{Search: true}) + if err == nil { + t.Fatal("a third search was admitted against a ration of two") + } + if got := reasonOf(t, err); got != ReasonSearches { + t.Errorf("reason = %q, want %q", got, ReasonSearches) + } + + // The point of a separate ration: everything else still works. + if err := l.Admit("alice", Admission{}); err != nil { + t.Errorf("an ordinary request was refused because searches ran out: %v", err) + } + + // And a refunded search is not a spent one. + l.Refund("alice", Admission{Search: true}) + if err := l.Admit("alice", Admission{Search: true}); err != nil { + t.Errorf("a refunded search ration was not returned: %v", err) + } +} + +func TestSearchRationIsPublishedWithTheOtherLimits(t *testing.T) { + lim := testLimits() + lim.DailySearches = 3 + l, done := open(t, t.TempDir(), lim) + defer done() + + if got := l.Status("alice", "anon").Limits.Searches; got != 3 { + t.Errorf("published search limit = %d, want 3 — a limit a caller cannot read is one they can only discover by hitting it", got) + } +} diff --git a/gateway/internal/server/proxy.go b/gateway/internal/server/proxy.go index a715f5f..0727391 100644 --- a/gateway/internal/server/proxy.go +++ b/gateway/internal/server/proxy.go @@ -116,7 +116,7 @@ func (s *Server) handleProxy(w http.ResponseWriter, r *http.Request) { // `deepseek status` free and safe in a loop against the free tier, // exactly as it is against the real API. billable := route.Format != policy.FormatNone - var reserve float64 + adm := quota.Admission{Search: decision.Search} if billable { if s.upstreamDry.Load() { // DeepSeek says the account is unusable. The local ledger's @@ -129,8 +129,8 @@ func (s *Server) handleProxy(w http.ResponseWriter, r *http.Request) { // forwarded. This is what makes the budget a ceiling: without the // reservation, every in-flight request is unbilled and the breaker // only notices after the money is spent. - reserve = meter.Estimate(decision.Model, len(decision.Body), decision.MaxTokens) - if err := s.ledger.Admit(subject, reserve); err != nil { + adm.ReserveUSD = meter.Estimate(decision.Model, len(decision.Body), decision.MaxTokens, decision.Search) + if err := s.ledger.Admit(subject, adm); err != nil { s.writeLimit(w, err) return } @@ -141,7 +141,7 @@ func (s *Server) handleProxy(w http.ResponseWriter, r *http.Request) { if err := s.acquire(r); err != nil { if billable { - s.ledger.Refund(subject, reserve) + s.ledger.Refund(subject, adm) } w.Header().Set("Retry-After", "5") writeError(w, http.StatusServiceUnavailable, typeQuota, @@ -153,7 +153,7 @@ func (s *Server) handleProxy(w http.ResponseWriter, r *http.Request) { s.stats.InFlight(1) defer s.stats.InFlight(-1) - s.forward(w, r, route, decision, subject, billable, reserve) + s.forward(w, r, route, decision, subject, billable, adm) } // edgeCountry reads the two-letter country the CDN attached to this @@ -194,7 +194,8 @@ func (s *Server) acquire(r *http.Request) error { } } -func (s *Server) forward(w http.ResponseWriter, r *http.Request, route policy.Route, d *policy.Decision, subject string, billable bool, reserve float64) { +func (s *Server) forward(w http.ResponseWriter, r *http.Request, route policy.Route, d *policy.Decision, subject string, billable bool, adm quota.Admission) { + reserve := adm.ReserveUSD url := strings.TrimRight(s.cfg.UpstreamBaseURL, "/") + route.Upstream var payload io.Reader @@ -204,7 +205,7 @@ func (s *Server) forward(w http.ResponseWriter, r *http.Request, route policy.Ro up, err := http.NewRequestWithContext(r.Context(), route.Method, url, payload) if err != nil { if billable { - s.ledger.Refund(subject, reserve) + s.ledger.Refund(subject, adm) } writeError(w, http.StatusInternalServerError, typeInternal, "could not build the upstream request") return @@ -223,7 +224,7 @@ func (s *Server) forward(w http.ResponseWriter, r *http.Request, route policy.Ro secret, fingerprint, err := s.keys.Next() if err != nil { if billable { - s.ledger.Refund(subject, reserve) + s.ledger.Refund(subject, adm) } s.writeLimit(w, "a.LimitError{Reason: quota.ReasonCredits}) return @@ -239,6 +240,7 @@ func (s *Server) forward(w http.ResponseWriter, r *http.Request, route policy.Ro } up.Header.Set("User-Agent", "dsgate") + sent := time.Now() resp, err := s.http.Do(up) if err != nil { if r.Context().Err() != nil { @@ -258,13 +260,20 @@ func (s *Server) forward(w http.ResponseWriter, r *http.Request, route policy.Ro // Never reached the model, so it cost nothing and the caller keeps // their request allowance. if billable { - s.ledger.Refund(subject, reserve) + s.ledger.Refund(subject, adm) } + s.stats.Upstream("unreachable", true, 0) writeError(w, http.StatusBadGateway, typeUpstream, "could not reach DeepSeek: "+err.Error()) return } defer resp.Body.Close() + // One observation per round trip, and only DeepSeek's own failures + // count as faults: a 4xx is the caller's request coming back. + s.stats.Upstream(strconv.Itoa(resp.StatusCode), + resp.StatusCode == http.StatusTooManyRequests || resp.StatusCode >= 500, + time.Since(sent)) + // A key that upstream refuses for money or validity is done, and // leaves the rotation now rather than after it has failed everyone // else's request too. Other 4xx are about the request, not the key. @@ -326,7 +335,7 @@ func (s *Server) forward(w http.ResponseWriter, r *http.Request, route policy.Ro if upstreamFault { // No tokens were generated and the fault was not the // caller's: give everything back. - s.ledger.Refund(subject, reserve) + s.ledger.Refund(subject, adm) } else { // The caller's own 4xx keeps its request debit, but the // money reserved for it goes back to the pool. @@ -491,6 +500,12 @@ func (s *Server) writeLimit(w http.ResponseWriter, err error) { w.Header().Set("Retry-After", "30") writeError(w, http.StatusServiceUnavailable, typeInternal, "the free tier is temporarily unavailable; retry shortly") + case quota.ReasonSearches: + // A distinct message because the fix is distinct: the rest of the + // tier still works, so "come back tomorrow" would be wrong. + retryAfter(w, lim.RetryAfter(time.Now())) + writeError(w, http.StatusTooManyRequests, typeQuota, + "you have used today's web-search allowance. Ordinary requests still work — searches reset at 00:00 UTC, or bring your own key for unlimited search: https://platform.deepseek.com/api_keys") case quota.ReasonDailyBudget: retryAfter(w, lim.RetryAfter(time.Now())) writeError(w, http.StatusTooManyRequests, typeQuota, diff --git a/gateway/internal/server/server_test.go b/gateway/internal/server/server_test.go index 35e4790..1e8c585 100644 --- a/gateway/internal/server/server_test.go +++ b/gateway/internal/server/server_test.go @@ -89,6 +89,7 @@ func newHarness(t *testing.T, up *upstream, tune func(*Config, *quota.Limits)) * DailyRequests: 5, DailyInputTokens: 10000, DailyOutputTokens: 5000, + DailySearches: 2, DailyBudgetUSD: 1, TotalBudgetUSD: 10, } @@ -901,3 +902,55 @@ func TestUpstreamDryBalanceStopsAdmissions(t *testing.T) { t.Errorf("HTTP %d after the account recovered, want 200", resp2.StatusCode) } } + +// web_search has to work end to end on the free tier — it is the reason +// `deepseek respond --web-search` exists — and it has to stay rationed, +// because one search costs about what ten ordinary turns cost. +func TestWebSearchIsCarriedAndRationed(t *testing.T) { + up := newUpstream(t, func(w http.ResponseWriter, r *http.Request) { + io.WriteString(w, chatReply(100, 50)) + }) + h := newHarness(t, up, nil) // DailySearches: 2 + tok := h.enrol(t) + + const search = `{"input":"who won","tools":[{"type":"web_search"}]}` + for i := 0; i < 2; i++ { + resp := h.do(t, "POST", "/responses", tok, search) + resp.Body.Close() + if resp.StatusCode != http.StatusOK { + t.Fatalf("search %d: status = %d, want 200", i+1, resp.StatusCode) + } + h.settle(t) + } + // The tool must reach DeepSeek intact — a gateway that quietly dropped + // it would return a confidently unsourced answer. + tools, _ := up.last(t).Body["tools"].([]any) + if len(tools) != 1 { + t.Fatalf("upstream saw %d tools, want the one that was sent", len(tools)) + } + if kind, _ := tools[0].(map[string]any)["type"].(string); kind != "web_search" { + t.Errorf("upstream saw tool type %q, want web_search", kind) + } + + resp := h.do(t, "POST", "/responses", tok, search) + defer resp.Body.Close() + if resp.StatusCode != http.StatusTooManyRequests { + t.Fatalf("a third search past a ration of two: status = %d, want 429", resp.StatusCode) + } + var body struct { + Error struct { + Type string `json:"type"` + Message string `json:"message"` + } `json:"error"` + } + json.NewDecoder(resp.Body).Decode(&body) + if !strings.Contains(body.Error.Message, "search") { + t.Errorf("the refusal does not say searches ran out: %q", body.Error.Message) + } + // Ordinary requests must survive an exhausted search ration. + plain := h.do(t, "POST", "/responses", tok, `{"input":"hi"}`) + defer plain.Body.Close() + if plain.StatusCode != http.StatusOK { + t.Errorf("an ordinary request was refused after searches ran out: status = %d", plain.StatusCode) + } +} diff --git a/gateway/internal/server/status.go b/gateway/internal/server/status.go index 48828f7..47a955f 100644 --- a/gateway/internal/server/status.go +++ b/gateway/internal/server/status.go @@ -42,6 +42,16 @@ type PublicStatus struct { Countries []stats.Count `json:"countries"` Top []TopSubject `json:"top_subjects"` + // History is the last 30 UTC days of traffic, oldest first. A service + // this size serves a few requests an hour at best, which is why the + // live five-minute view is nearly always zero and cannot be the whole + // story — the daily series is where the traffic is actually visible. + History []quota.Day `json:"history"` + + // Upstream is what DeepSeek has done for us lately, so a visitor can + // tell our outage from theirs without leaving the page. + Upstream stats.Upstream `json:"upstream"` + Keys PoolStatus `json:"key_pool"` Limits quota.UserCaps `json:"daily_limits_per_user"` System stats.System `json:"system"` @@ -107,6 +117,11 @@ func withoutMoney(t quota.Totals) quota.Totals { return t } +// historyDays is how far the daily series goes back. Thirty days is one +// screen of bars at a readable width, and long enough that a week of +// growth or a quiet stretch is legible rather than a rounding error. +const historyDays = 30 + // publicStatusTTL caches the document. The dashboard polls, several // people may have it open, and every field is a five-minute rolling // figure — recomputing per request would spend more CPU on watching the @@ -171,6 +186,8 @@ func (s *Server) buildStatus() *PublicStatus { SubjectsToday: h.Subjects, }, Live: snap.Live, + History: s.ledger.History(historyDays), + Upstream: snap.Upstream, Endpoints: snap.Endpoints, Countries: snap.Countries, Top: rows, diff --git a/gateway/internal/server/status_test.go b/gateway/internal/server/status_test.go index 2cfe6f9..4dbc548 100644 --- a/gateway/internal/server/status_test.go +++ b/gateway/internal/server/status_test.go @@ -165,7 +165,7 @@ func TestStatusReportsExhaustion(t *testing.T) { // Spend the pool for real rather than configuring it to zero: the // state has to follow actual spend, which is the thing that goes // wrong in production. - h.ledger.Admit("someone", 0) + h.ledger.Admit("someone", quota.Admission{}) h.ledger.Charge("someone", "chat", "deepseek-v4-flash", 10, 0, 10, 0.002, 0, false) st := getStatus(t, h) @@ -339,3 +339,106 @@ func TestEmptyPoolIsAnHonest402(t *testing.T) { t.Errorf("state = %q, want %q", st.State, StateDry) } } + +// The daily series is the dashboard's main chart, so it has to be a +// calendar — every day present, quiet days as zeroes — and it has to keep +// the same secret the rest of the document keeps: no money. +func TestStatusHistoryIsACalendarWithoutMoney(t *testing.T) { + up := newUpstream(t, func(w http.ResponseWriter, r *http.Request) { + io.WriteString(w, chatReply(120, 80)) + }) + h := newHarness(t, up, nil) + tok := h.enrol(t) + h.do(t, "POST", "/chat/completions", tok, `{"messages":[]}`).Body.Close() + h.settle(t) + + resp := h.do(t, "GET", "/v1/status", "", "") + defer resp.Body.Close() + raw, _ := io.ReadAll(resp.Body) + + var st PublicStatus + if err := json.Unmarshal(raw, &st); err != nil { + t.Fatal(err) + } + if len(st.History) != historyDays { + t.Fatalf("history has %d days, want %d — gaps in a chart read as zero traffic, not as missing data", len(st.History), historyDays) + } + last := st.History[len(st.History)-1] + if last.Requests != 1 { + t.Errorf("today's row shows %d requests, want the 1 just served", last.Requests) + } + if last.Subjects != 1 { + t.Errorf("today's row shows %d subjects, want 1", last.Subjects) + } + // Dates must be ordered oldest first, or the chart draws backwards. + for i := 1; i < len(st.History); i++ { + if st.History[i-1].Date >= st.History[i].Date { + t.Fatalf("history is not oldest-first at %d: %q then %q", i, st.History[i-1].Date, st.History[i].Date) + } + } + // A per-day spend series would be a map of when we are cheapest to + // empty. It must not be in the public document at any nesting. + var probe struct { + History []map[string]any `json:"history"` + } + json.Unmarshal(raw, &probe) + for _, day := range probe.History { + for k := range day { + if strings.Contains(k, "usd") || strings.Contains(k, "spend") || strings.Contains(k, "cost") { + t.Errorf("the daily series carries money: %q", k) + } + } + } +} + +// "Is it you or DeepSeek" is the question a visitor actually has when a +// request fails, so the answer has to be on the page — and it must not +// claim health it has not observed. +func TestStatusReportsUpstreamHealth(t *testing.T) { + var fail bool + up := newUpstream(t, func(w http.ResponseWriter, r *http.Request) { + if fail { + w.WriteHeader(http.StatusBadGateway) + return + } + io.WriteString(w, chatReply(10, 10)) + }) + h := newHarness(t, up, nil) + tok := h.enrol(t) + + // Before anything has been asked of DeepSeek, the honest answer is + // "unknown" rather than a cheerful all-clear. + var st PublicStatus + resp := h.do(t, "GET", "/v1/status", "", "") + json.NewDecoder(resp.Body).Decode(&st) + resp.Body.Close() + if st.Upstream.State != "unknown" { + t.Errorf("upstream state before any call = %q, want unknown", st.Upstream.State) + } + + h.do(t, "POST", "/chat/completions", tok, `{"messages":[]}`).Body.Close() + h.settle(t) + h.invalidateStatus() + resp = h.do(t, "GET", "/v1/status", "", "") + json.NewDecoder(resp.Body).Decode(&st) + resp.Body.Close() + if st.Upstream.State != "ok" || st.Upstream.Calls != 1 { + t.Errorf("after one good call: state=%q calls=%d, want ok/1", st.Upstream.State, st.Upstream.Calls) + } + + fail = true + for i := 0; i < 3; i++ { + h.do(t, "POST", "/chat/completions", tok, `{"messages":[]}`).Body.Close() + h.settle(t) + } + h.invalidateStatus() + resp = h.do(t, "GET", "/v1/status", "", "") + json.NewDecoder(resp.Body).Decode(&st) + resp.Body.Close() + if st.Upstream.State != "down" { + t.Errorf("after three consecutive 502s: state=%q, want down", st.Upstream.State) + } + if st.Upstream.LastFault != "502" { + t.Errorf("last fault = %q, want 502", st.Upstream.LastFault) + } +} diff --git a/gateway/internal/server/web/app.js b/gateway/internal/server/web/app.js index 119a9c0..c63a6e1 100644 --- a/gateway/internal/server/web/app.js +++ b/gateway/internal/server/web/app.js @@ -107,23 +107,30 @@ pill.className = "pill " + cls; } - /* ---------- sparkline chart ---------- */ + /* ---------- daily chart ---------- + + Bars, one per UTC day, for the last thirty days. It used to be output + tokens per second over five minutes, which was the wrong instrument + for this service: a shared pool serving a few requests an hour is + idle almost every second, so the line was flat at zero whenever + anyone looked and said nothing true about whether the thing works. + A day is the smallest bucket that is usually non-empty here. */ var chart = (function () { var canvas = $("spark"); var ctx = canvas.getContext("2d"); - var target = []; // latest series from the server - var shown = []; // what is currently drawn (tweens toward target) + var days = []; // [{date, requests, input_tokens, output_tokens, subjects}] + var shown = []; // bar heights currently drawn (tween toward days) var tweenFrom = null; var tweenStart = 0; var raf = 0; var hoverIdx = -1; - var colors = { line: "#00c2e9", grid: "#9a9a9a" }; + var colors = { bar: "#00c2e9", grid: "#9a9a9a" }; var TWEEN_MS = 280; function refreshColors() { var cs = getComputedStyle(canvas); - colors.line = cs.color; + colors.bar = cs.color; colors.grid = cs.borderTopColor || cs.borderColor || colors.grid; } @@ -142,13 +149,11 @@ function draw() { var dim = size(); var w = dim.w, h = dim.h; - var pad = 4; + var pad = 6; ctx.clearRect(0, 0, w, h); - var s = shown; - var n = s.length; - - // baseline + // baseline: drawn even with no data, so an empty chart still reads + // as an axis rather than as a failed render. ctx.globalAlpha = 0.5; ctx.strokeStyle = colors.grid; ctx.lineWidth = 1; @@ -158,40 +163,32 @@ ctx.stroke(); ctx.globalAlpha = 1; - if (n < 2) return; + var n = shown.length; + if (n === 0) return; var max = 1; - for (var i = 0; i < n; i++) if (s[i] > max) max = s[i]; - - function x(i) { return (i / (n - 1)) * w; } - function y(v) { return h - pad - (v / max) * (h - pad * 2); } - - // area fill - ctx.beginPath(); - ctx.moveTo(0, h); - for (var j = 0; j < n; j++) ctx.lineTo(x(j), y(s[j])); - ctx.lineTo(w, h); - ctx.closePath(); - ctx.globalAlpha = 0.16; - ctx.fillStyle = colors.line; - ctx.fill(); - ctx.globalAlpha = 1; - - // line - ctx.beginPath(); - for (var k = 0; k < n; k++) { - if (k === 0) ctx.moveTo(x(k), y(s[k])); - else ctx.lineTo(x(k), y(s[k])); + for (var i = 0; i < n; i++) if (shown[i] > max) max = shown[i]; + + var slot = w / n; + var bw = Math.max(2, Math.min(slot - 2, 18)); + + for (var j = 0; j < n; j++) { + var cx = slot * (j + 0.5); + var v = shown[j]; + var bh = (v / max) * (h - pad - 1); + var isToday = j === n - 1; + // A zero day still gets a one-pixel tick. Nothing drawn at all + // looks like missing data; a floor line reads as a quiet day. + if (bh < 1) bh = v > 0 ? 1.5 : 1; + ctx.globalAlpha = v > 0 ? (isToday ? 1 : 0.75) : 0.22; + ctx.fillStyle = colors.bar; + ctx.fillRect(Math.round(cx - bw / 2), h - 1 - bh, Math.round(bw), bh); } - ctx.strokeStyle = colors.line; - ctx.lineWidth = 2; - ctx.lineJoin = "round"; - ctx.stroke(); + ctx.globalAlpha = 1; - // hover crosshair if (hoverIdx >= 0 && hoverIdx < n) { - var hx = x(hoverIdx); - ctx.globalAlpha = 0.6; + var hx = slot * (hoverIdx + 0.5); + ctx.globalAlpha = 0.5; ctx.strokeStyle = colors.grid; ctx.lineWidth = 1; ctx.beginPath(); @@ -199,41 +196,57 @@ ctx.lineTo(hx + 0.5, h); ctx.stroke(); ctx.globalAlpha = 1; - ctx.beginPath(); - ctx.arc(hx, y(s[hoverIdx]), 3.5, 0, Math.PI * 2); - ctx.fillStyle = colors.line; - ctx.fill(); } } function step(ts) { var t = Math.min(1, (ts - tweenStart) / TWEEN_MS); var e = 1 - Math.pow(1 - t, 3); // ease-out cubic - for (var i = 0; i < target.length; i++) { + for (var i = 0; i < days.length; i++) { var from = tweenFrom[i] || 0; - shown[i] = from + (target[i] - from) * e; + shown[i] = from + (days[i].requests - from) * e; } - shown.length = target.length; + shown.length = days.length; draw(); if (t < 1) raf = requestAnimationFrame(step); } - function update(series) { - if (!Array.isArray(series)) series = []; + function readout(i) { + var d = days[i]; + if (!d) return ""; + var when = d.date; + if (i === days.length - 1) when += " (today)"; + return when + " · " + fmtInt(d.requests) + " req · " + + fmtCompact(d.output_tokens) + " out · " + fmtInt(d.subjects) + + (d.subjects === 1 ? " person" : " people"); + } + + function update(history) { + if (!Array.isArray(history)) history = []; var clean = []; - for (var i = 0; i < series.length; i++) { - var v = Number(series[i]); - clean.push(isFinite(v) && v > 0 ? v : 0); + for (var i = 0; i < history.length; i++) { + var row = history[i] || {}; + var v = Number(row.requests); + clean.push({ + date: String(row.date || ""), + requests: isFinite(v) && v > 0 ? v : 0, + input_tokens: Number(row.input_tokens) || 0, + output_tokens: Number(row.output_tokens) || 0, + subjects: Number(row.subjects) || 0 + }); } var prevShown = shown.slice(); - target = clean; + days = clean; + if (clean.length) setText("chart-x0", clean[0].date); if (raf) cancelAnimationFrame(raf); + var heights = clean.map(function (d) { return d.requests; }); if (reducedMotion.matches || prevShown.length === 0) { - shown = clean.slice(); + shown = heights; draw(); return; } - // align previous frame to the new series length (both end at "now") + // Both series end at today, so align them from the right: yesterday + // stays yesterday when a new day appears on the end. tweenFrom = []; var shift = clean.length - prevShown.length; for (var j = 0; j < clean.length; j++) { @@ -245,14 +258,12 @@ } canvas.addEventListener("pointermove", function (ev) { - var n = shown.length; - if (n < 2) return; + var n = days.length; + if (n === 0) return; var rect = canvas.getBoundingClientRect(); var frac = (ev.clientX - rect.left) / rect.width; - hoverIdx = Math.max(0, Math.min(n - 1, Math.round(frac * (n - 1)))); - var age = n - 1 - hoverIdx; - setText("spark-readout", Math.round(shown[hoverIdx]) + " tok/s · " + - (age === 0 ? "now" : age + "s ago")); + hoverIdx = Math.max(0, Math.min(n - 1, Math.floor(frac * n))); + setText("spark-readout", readout(hoverIdx)); draw(); }); canvas.addEventListener("pointerleave", function () { @@ -372,6 +383,55 @@ } setInterval(tickCountdown, 1000); + /* ---------- upstream health ---------- + + Two rows, because a visitor whose request just failed has exactly one + question and it is not "what is your p99": is this you or DeepSeek? + Our own row is derived from the gateway's own state word; the DeepSeek + row is what our last calls to api.deepseek.com actually did. */ + + var UP_STATES = { + ok: ["reachable", "ok"], + degraded: ["some calls failing", "warn"], + down: ["not answering us", "crit"], + unknown: ["not called yet", "idle"] + }; + + function setDot(id, cls) { + var el = $(id); + if (el) el.className = "health-dot " + cls; + } + + function renderUpstream(d) { + // Ours: anything that still serves requests is working, and the two + // exhausted states are our limit rather than a fault. + var st = String(d.state || ""); + var usText = "serving requests", usCls = "ok"; + if (st === "degraded") { usText = "refusing — cannot record spend"; usCls = "crit"; } + else if (st === "day_exhausted") { usText = "today's budget spent"; usCls = "warn"; } + else if (st === "credit_exhausted") { usText = "credit pool empty"; usCls = "warn"; } + else if (st === "busy") { usText = "busy — requests queue"; usCls = "warn"; } + else if (st !== "operational") { usText = String(d.state || "unknown"); usCls = "idle"; } + setText("up-us", usText); + setDot("up-us-dot", usCls); + + var u = d.upstream || {}; + var m = UP_STATES[u.state] || UP_STATES.unknown; + setText("up-ds", m[0]); + setDot("up-ds-dot", m[1]); + + var bits = []; + if (u.latency_ms > 0) bits.push("last good call " + fmtInt(u.latency_ms) + " ms"); + if (u.last_ok_ago_sec >= 0) bits.push(fmtDur(u.last_ok_ago_sec) + " ago"); + if (u.fault_streak > 0) { + bits.push(fmtInt(u.fault_streak) + " failing in a row" + + (u.last_fault ? " (" + u.last_fault + ")" : "")); + } else if (u.faults > 0) { + bits.push(fmtInt(u.faults) + " of " + fmtInt(u.calls) + " calls failed since boot"); + } + setText("up-note", bits.length ? bits.join(" · ") : "nothing forwarded since this process started"); + } + /* ---------- render ---------- */ function render(d) { @@ -387,19 +447,29 @@ setText("t-flight", fmtInt(live.in_flight)); var usage = d.usage || {}; + var today = usage.today || {}; + var life = usage.lifetime || {}; setText("t-today", fmtInt(usage.subjects_today)); + setText("t-req-today", fmtInt(today.requests)); + setText("t-req-life", fmtCompact(life.requests)); var sys = d.system || {}; setText("t-uptime", fmtDur(sys.uptime_sec)); - chart.update(live.series); + var history = Array.isArray(d.history) ? d.history : []; + var sum30 = 0; + history.forEach(function (row) { sum30 += Number(row && row.requests) || 0; }); + setText("t-req-30d", fmtCompact(sum30)); + chart.update(history); + setText("chart-now", "right now: " + fmtRate(live.tokens_per_sec) + + " tokens/sec · " + fmtInt(live.in_flight) + " in flight"); + + renderUpstream(d); var credit = d.credit || {}; setGauge("g-day", credit.day_remaining_pct); setGauge("g-pool", credit.pool_remaining_pct); - var today = usage.today || {}; - var life = usage.lifetime || {}; setText("to-req", fmtCompact(today.requests)); setText("lt-req", fmtCompact(life.requests)); setText("to-in", fmtCompact(today.input_tokens)); @@ -430,6 +500,7 @@ setText("lim-req", fmtInt(lim.requests)); setText("lim-in", fmtCompact(lim.input_tokens)); setText("lim-out", fmtCompact(lim.output_tokens)); + setText("lim-search", fmtInt(lim.searches)); var rAt = Date.parse(d.resets_at); var sNow = Date.parse(d.now); diff --git a/gateway/internal/server/web/index.html b/gateway/internal/server/web/index.html index 51a65ea..e5c51bf 100644 --- a/gateway/internal/server/web/index.html +++ b/gateway/internal/server/web/index.html @@ -74,21 +74,21 @@

Live status

-
—tokens/sec
-
—requests/min
-
—live users (5m)
-
—in flight
-
—users today
+
—requests today
+
—people today
+
—requests · 30d
+
—requests · all time
—uptime
- output tokens/sec — last 5 minutes + requests per day — last 30 days
- - + + +

—

@@ -126,14 +126,50 @@

Totals

+
+

Is it us or DeepSeek?

+
    +
  • + + this gateway + — +
    • +
  • + + DeepSeek upstream + — +
    • +
+

—

+

Measured from our own calls to + api.deepseek.com, not from a status page — a page cannot see + a route that is broken only from here. DeepSeek's own + incident page is the place to + check for anything wider.

+

Endpoints

  • no data yet
+
+ +

Countries

  • no data yet
+
+

Right now

+
    +
  • in flight —
    • +
  • tokens/sec —
    • +
  • requests/min —
    • +
  • active (5m) —
    • +
+

A pool this size is idle most of the day, so these + are usually zero. That is the honest reading, not a broken widget — the + daily chart above is where the traffic shows.

+
@@ -186,7 +222,13 @@

Per-user daily limits

  • 30 requests
  • 60k input tokens
  • 20k output tokens
    • +
  • 3 web searches
    • +

    Server-side web_search works here: + deepseek respond "…" --web-search, or the web_search + tool on /responses from any OpenAI client. It has its own small + ration because one search reads whole pages into the prompt and costs about + what ten ordinary turns cost — see economics.

    diff --git a/gateway/internal/server/web/pages/economics.html b/gateway/internal/server/web/pages/economics.html index 2048ebb..d2b583c 100644 --- a/gateway/internal/server/web/pages/economics.html +++ b/gateway/internal/server/web/pages/economics.html @@ -57,8 +57,51 @@

    The same task

    Not twenty percent cheaper. Thirty to a few hundred times cheaper, depending on the model and how much of the prompt caches.

    +

    That table is still arithmetic on a rate card, which is the weakest kind of cost claim — it assumes every model spends the same tokens on the same work, and they do not. The section below is the stronger evidence: what independent benchmarks actually paid.

    + +

    What the benchmarks actually paid

    +

    Two leaderboards publish a dollar figure next to the score, measured from their own runs rather than multiplied out of a price list. Artificial Analysis reports cost per task — total spend to run its Intelligence Index divided by the number of tasks — alongside the index score itself:

    + + + + + + + + + + + + +
    modelintelligence indexcost per task
    deepseek-v4-flash (max)52$0.03
    deepseek-v4-pro (max)45$0.05
    GPT-5.6 Luna (max)52$0.05
    Gemini 3.6 Flash52$0.56
    GPT-5.6 Sol (max)61$1.23
    Claude Sonnet 5 (max)55$1.72
    Claude Opus 5 (max)63$2.34
    Claude Fable 562$3.14
    +

    The same source publishes what the whole index cost to run per model, which is the least ambiguous number on this page: $72.03 for deepseek-v4-flash against $3,836.05 for Claude Opus 5 and $5,455.22 for Claude Fable 5 — 53× and 76× on identical work.

    + +

    On coding specifically, SWE-rebench runs one fixed harness over 111 real GitHub issues and publishes cost per problem next to the resolve rate:

    + + + + + + + + + + +
    modelresolvedcost per problem
    Claude Fable 5 (high)64.5%$4.40
    Claude Opus 5 (high)63.4%$3.47
    GPT-5.6 Sol (medium)62.3%$0.85
    Claude Sonnet 5 (high)56.8%$1.43
    GPT-5.6 Luna (medium)43.6%$0.11
    deepseek-v4-pro (high)40.2%$0.15
    +

    And one independent developer ran the same build prompt three times back to back in early August 2026 and published the receipts: Claude Opus 5 $0.3183, Kimi K3 $0.2485, deepseek-v4-flash $0.0049. All three produced working output; DeepSeek failed silently on image handling where Claude improvised around bugs that stopped the others.

    + +

    Where the cheap-token argument breaks

    +

    Two numbers in those tables cut against the story this page is telling, and leaving them out would make it propaganda. On Artificial Analysis, GPT-5.6 Luna scores the same 52 as flash for $0.05 against flash's $0.03 — a 1.7× gap, not a hundredfold one. On SWE-rebench, Luna beats deepseek-v4-pro on both axes at once: more issues resolved, less money per issue. So the honest shape of the claim is narrower than "DeepSeek is the cheapest way to get work done":

    +
      +
    • Against the flagship tier, the gap is real and enormous. One to two orders of magnitude per task, on published same-harness runs, confirmed by two independent measurers.
      • +
    • Against the cheap frontier tier, the gap mostly evaporates. The cheap tiers of the big labs now land within a factor of two of DeepSeek's per-task cost, sometimes with better scores.
      • +
    • The top of the quality range is not for sale at this price. Flash's 52 sits ten-plus index points below Opus 5 and Fable 5, and Artificial Analysis pairs that $0.03 with a 37% score on Humanity's Last Exam and an 84% hallucination rate. Paying 76× more buys something; whether it buys enough for your task is your measurement to make, not ours.
      • +
    +

    Every figure above was read on 2026-08-07 and they drift for two reasons at once: vendors reprice (DeepSeek's own cost-to-run fell from $1,071 to $176 for v4-pro after a price cut), and benchmarks reweight their indices. Treat the ratios as this month's, not as constants.

    +

    What this does not claim

    -

    Per-token is not per-task. A stronger model that solves a hard problem in one attempt can beat a cheaper model that needs three, and on the hardest work the frontier models earn their price. Tokenizers differ too — Anthropic notes that its current tokenizer produces roughly 30% more tokens for the same text than its previous one, so identical work is not identical token counts across vendors. And DeepSeek has announced peak-hour pricing at 2× the listed rates, effective date not yet set. The honest claim is narrower and still remarkable: for the broad middle of real work — summarize, translate, refactor, answer, glue — the going rate differs by two orders of magnitude depending on whose API you call.

    +

    Three smaller caveats on top of the big one. Tokenizers differ, so identical text is not identical token counts across vendors — Anthropic notes its current tokenizer emits roughly 30% more tokens for the same input than its previous one. DeepSeek has announced peak-hour pricing at 2× the listed rates, with no effective date yet, which would halve the gap during working hours in Beijing. And a harness costs money too: one comparison held flash constant across four agent frameworks and watched cost per successful task swing 2.7× — $0.073 on the cheapest, $0.195 on Claude Code — so the tool you drive the model with can matter as much as the model.

    +

    What survives all of that is still worth saying plainly: for the broad middle of real work — summarize, translate, refactor, answer, glue — the going rate differs by one to two orders of magnitude depending on whose API you call, and the cheapest credible option is no longer a toy.

    Why it matters

    Chat is measured in thousands of tokens; agents are measured in millions. The moment a model works unattended — reading files, retrying, checking its own output — token consumption stops tracking human attention and starts tracking machine patience. An overnight agent run that emits ten million output tokens costs $2.80 at flash prices and $500 at Fable prices. One of those is "leave it running"; the other is a line item that gets a meeting. At frontier prices, autonomy is a luxury good. At flash prices, it is a background process.

    diff --git a/gateway/internal/server/web/style.css b/gateway/internal/server/web/style.css index b0e71a0..c8cd708 100644 --- a/gateway/internal/server/web/style.css +++ b/gateway/internal/server/web/style.css @@ -195,7 +195,7 @@ h3 { font-size: 13px; margin: 0 0 10px; text-transform: uppercase; letter-spacin .tiles { display: grid; - grid-template-columns: repeat(6, 1fr); + grid-template-columns: repeat(5, 1fr); gap: 10px; margin-bottom: 14px; } @@ -234,7 +234,7 @@ h3 { font-size: 13px; margin: 0 0 10px; text-transform: uppercase; letter-spacin #spark { display: block; width: 100%; - height: 150px; + height: 168px; color: var(--cyan); /* line + fill colour, read by app.js */ border-color: var(--muted); /* axis/grid colour, read by app.js */ touch-action: pan-y; @@ -247,6 +247,47 @@ h3 { font-size: 13px; margin: 0 0 10px; text-transform: uppercase; letter-spacin margin-top: 4px; } +/* the live figures, demoted under the chart: true but usually zero */ +.chart-now { + margin: 6px 0 0; + font-size: 11.5px; + color: var(--muted); + font-variant-numeric: tabular-nums; +} + +/* upstream health rows */ +.health { list-style: none; margin: 0 0 10px; padding: 0; } +.health li { + display: grid; + grid-template-columns: 10px 1fr auto; + align-items: center; + gap: 10px; + padding: 7px 0; + border-bottom: 1px solid var(--line); +} +.health li:last-child { border-bottom: 0; } +.health-who { color: var(--text); font-size: 13px; } +.health-state { + color: var(--muted); + font-size: 12.5px; + text-align: right; + font-variant-numeric: tabular-nums; +} +.health-dot { + width: 8px; + height: 8px; + border-radius: 50%; + background: var(--muted); + box-shadow: 0 0 0 3px color-mix(in srgb, var(--muted) 18%, transparent); +} +/* Colour is a second signal here, never the only one: the words next to + each dot say the same thing, so the panel survives being read in + greyscale or by someone who cannot separate red from green. */ +.health-dot.ok { background: var(--cyan); box-shadow: 0 0 0 3px color-mix(in srgb, var(--cyan) 20%, transparent); } +.health-dot.warn { background: var(--yellow); box-shadow: 0 0 0 3px color-mix(in srgb, var(--yellow) 20%, transparent); } +.health-dot.crit { background: var(--pink); box-shadow: 0 0 0 3px color-mix(in srgb, var(--pink) 20%, transparent); } +.health-dot.idle { opacity: 0.55; } + /* two-column card rows */ .cols { display: grid; diff --git a/gateway/internal/stats/stats.go b/gateway/internal/stats/stats.go index 7454bf2..f9977f2 100644 --- a/gateway/internal/stats/stats.go +++ b/gateway/internal/stats/stats.go @@ -47,6 +47,27 @@ type bucket struct { output int64 } +// Upstream is what the gateway has observed of DeepSeek itself. +type Upstream struct { + // State is one word: ok, degraded (some calls failing) or down + // (nothing has succeeded since the last failure). + State string `json:"state"` + // LastOKAgoSec and LastFaultAgoSec are ages rather than timestamps so + // a reader does not have to trust our clock against theirs. Negative + // means it has not happened in this process's lifetime. + LastOKAgoSec int64 `json:"last_ok_ago_sec"` + LastFaultAgoSec int64 `json:"last_fault_ago_sec"` + // LastFault is the shape of the last failure — a status code or a + // transport error class. Never a URL, a key or a prompt. + LastFault string `json:"last_fault,omitempty"` + Calls int64 `json:"calls"` + Faults int64 `json:"faults"` + // FaultStreak is consecutive failures right now. It is the number that + // distinguishes "one bad request" from "DeepSeek is having an outage". + FaultStreak int `json:"fault_streak"` + LatencyMS int64 `json:"latency_ms"` +} + // Collector accumulates the live view. type Collector struct { mu sync.Mutex @@ -66,6 +87,19 @@ type Collector struct { inFlight int64 started time.Time + // Upstream health: what DeepSeek did the last time we spoke to it. + // This is the honest answer to "is it you or them", and it is a + // first-party observation rather than a status page — a vendor's + // status page cannot see a route that is broken only from here, and + // DeepSeek's is not reachable from outside China anyway. + upLastOK time.Time + upLastFault time.Time + upFaultWhat string + upCalls int64 + upFaults int64 + upStreak int + upLatencyMS int64 + now func() time.Time } @@ -191,10 +225,11 @@ type System struct { // Snapshot is everything the dashboard needs from this package. type Snapshot struct { - Live Live `json:"live"` - Endpoints []Count `json:"endpoints"` - Countries []Count `json:"countries"` - System System `json:"system"` + Live Live `json:"live"` + Endpoints []Count `json:"endpoints"` + Countries []Count `json:"countries"` + System System `json:"system"` + Upstream Upstream `json:"upstream"` } // Snapshot reads the live view. Cheap enough to call per request, but the @@ -234,6 +269,7 @@ func (c *Collector) Snapshot() Snapshot { }, Endpoints: topLocked(c.endpoints, 10), Countries: topLocked(c.countries, 12), + Upstream: c.upstreamLocked(), System: System{ UptimeSec: int64(c.now().Sub(c.started).Seconds()), Load1: loadAvg1(), @@ -297,3 +333,63 @@ func loadAvg1() float64 { } return 0 } + +// Upstream records one round trip to DeepSeek. +// +// fault is the caller's judgement, not a status code test, because the +// gateway already distinguishes "DeepSeek is broken" (429, 5xx, transport +// failure) from "the caller's own request came back 4xx" — and only the +// first is an upstream fault. Counting a user's malformed JSON as a +// DeepSeek outage would make this panel lie in the most misleading +// direction. +func (c *Collector) Upstream(what string, fault bool, latency time.Duration) { + c.mu.Lock() + defer c.mu.Unlock() + now := c.now() + c.upCalls++ + if fault { + c.upFaults++ + c.upStreak++ + c.upLastFault = now + c.upFaultWhat = what + return + } + c.upStreak = 0 + c.upLastOK = now + if ms := latency.Milliseconds(); ms >= 0 { + c.upLatencyMS = ms + } +} + +// upstreamLocked reduces the counters to the shape the dashboard reads. +func (c *Collector) upstreamLocked() Upstream { + now := c.now() + ago := func(t time.Time) int64 { + if t.IsZero() { + return -1 + } + return int64(now.Sub(t).Seconds()) + } + u := Upstream{ + State: "ok", + LastOKAgoSec: ago(c.upLastOK), + LastFaultAgoSec: ago(c.upLastFault), + LastFault: c.upFaultWhat, + Calls: c.upCalls, + Faults: c.upFaults, + FaultStreak: c.upStreak, + LatencyMS: c.upLatencyMS, + } + switch { + case c.upCalls == 0: + // Nothing has been asked of DeepSeek since this process started, so + // there is nothing to report either way. Saying "ok" would be a + // guess dressed as a measurement. + u.State = "unknown" + case c.upStreak >= 3: + u.State = "down" + case c.upStreak > 0: + u.State = "degraded" + } + return u +} From 84845504fa26a81d4f232baa4bad69f6793535cf Mon Sep 17 00:00:00 2001 From: Eric Wang Date: Fri, 7 Aug 2026 05:22:19 -0700 Subject: [PATCH 2/3] feat(site): a web-search toggle in the playground MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The gateway now carries DeepSeek's server-side web_search, so the browser playground should offer it too — it is a first-class client on the same gateway and quota, not a demo. The control only exists on the Responses format, because that is the only format DeepSeek offers the tool on; a checkbox that silently did nothing on the other three would be worse than no checkbox. The equivalent-command panel gains --web-search so the copy-paste stays honest, and the note says what a search costs against the daily ration. Markup lives in build.py, which generates the page; editing the generated HTML directly leaves the two disagreeing and build.py --check catches it. Co-Authored-By: Claude Fable 5 --- site/build.py | 10 ++++++++++ site/playground.js | 11 +++++++++-- site/playground/index.html | 10 ++++++++++ site/style.css | 11 +++++++++++ 4 files changed, 40 insertions(+), 2 deletions(-) diff --git a/site/build.py b/site/build.py index 8c97450..3061697 100644 --- a/site/build.py +++ b/site/build.py @@ -1405,6 +1405,16 @@ def jstr(s):

    + +

    + +