diff --git a/AGENTS.md b/AGENTS.md index 781eb7b..7ff5c3d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -155,9 +155,16 @@ deepseek free off # forget the enrolment on this machine ``` Free-tier limits, per UTC day: 30 requests, 60K input tokens, 20K output -tokens, 4K output per call, 128KB per request body, `deepseek-v4-flash` -only. A request for pro is **refused, not downgraded**. `models` and -`status` cost no quota; everything that can generate a token does. +tokens, 3 web searches, 4K output per call, 128KB per request body, +`deepseek-v4-flash` only. A request for pro is **refused, not +downgraded**. `models` and `status` cost no quota; everything that can +generate a token does. + +`respond --web-search` works on the free tier and spends one of the three +daily searches. It is rationed that tightly because DeepSeek reads whole +pages into the prompt — one measured search request billed 40K input +tokens, about ten ordinary turns — so treat it as a few lookups a day, not +a research loop. Other server-side tools are still refused. Errors from the gateway carry `"type":"free_tier_*"` and a message that already contains the next step — do not append DeepSeek's own advice to diff --git a/TASTE.md b/TASTE.md index df428dc..5d53d4c 100644 --- a/TASTE.md +++ b/TASTE.md @@ -141,6 +141,24 @@ a human always has somewhere to go. **Expires.** If DeepSeek publishes a documented status JSON endpoint, consume it and report both. +**2026-08-07 — expiry tested, still holds, and now for a second reason.** +Probed for a JSON endpoint from the production host: `/api/v2/summary.json`, +`/api/v2/status.json`, `/api/status`, `/api/v1/status` and the bare page all +fail. `status.deepseek.com` resolves to an Aliyun Beijing load balancer +(`statuspage.flashcat.cloud`) that accepts the TCP connection and then never +completes the TLS handshake from outside China, while `api.deepseek.com` +answers fine from the same box. So the page is both unparseable and +unreachable from where our gateway runs. Consuming it would mean shipping a +health signal that is permanently unknown — worse than the wrong all-clear +this scar was written about, because it would also be *our* dashboard showing +red for *their* geography. What the gateway publishes instead is a +first-party observation: every round trip to `api.deepseek.com` is recorded, +and `/v1/status` carries an `upstream` block with the last success, the +consecutive-failure streak and the last fault shape. That answers the +question a visitor actually has — is this you or them — with data a status +page structurally cannot have, since it cannot see a route that is broken +only from here. The incident page stays linked for humans. + --- ## 2026-08-05 rejected: shipping the tokenizer, or estimating from the ratios @@ -345,3 +363,50 @@ vectors by coincidence and fail against the gateway every time. **Expires.** If the CLI and the gateway ever merge into one module, the Go halves should share code and only the browser stays separate. + +--- + +## 2026-08-07 accepted, after a measurement: `web_search` on the free tier + +Recorded here because it reverses an earlier refusal, and a reversal +without its evidence is just a mood swing. + +**The refusal it replaces.** `policy.forbidServerTools` rejected every +server-side tool with a real argument: such a tool "performs billed work +that never appears in the usage object, which would put its cost outside +every ceiling this gateway enforces". Correct reasoning, untested premise. + +**What the measurement showed.** One `respond --web-search` call against +the live API on 2026-08-07 made eleven server-side calls (searches, page +opens, an in-page find) and reported 40,260 input tokens, 32,000 of them +cache hits, 3,100 output. The account balance moved by nothing beyond +those tokens — eleven searches at a frontier vendor's $10-per-1,000 rate +would have been $0.11, or 0.79 CNY, and unmistakable against a 14.26 CNY +balance. So the premise was wrong in the half that mattered: there is no +per-search fee, and the entire cost of a search arrives as input tokens +that this gateway already meters exactly. + +**What the measurement did break.** The half of the premise that was +right, in a different place than expected. `meter.Estimate` bounded a +request's input at one token per body byte — true for every other request +and false for this one, because DeepSeek chooses how many pages to read. +A search request's input is upstream-controlled, so the admission +reservation no longer bounds it from the body. + +**Reuse.** Allow `web_search` (both the bare and dated tool names), refuse +every other server-side tool still, and pay for the change in two places: +a 256k-token input allowance in the reservation, roughly six times the +observed case; and a per-user daily ration of three searches, which is a +new quota dimension because one search costs about what ten ordinary turns +cost and the request counter alone would let one caller quietly take a +quarter of the day's budget. Stating the trade honestly: within the +allowance the budget is still a hard ceiling, and beyond it a search can +overshoot by the difference, bounded by the few distinct callers who can +be mid-search at once given a per-subject in-flight cap of one. + +**Expires.** If DeepSeek starts billing searches separately, or documents +a cap on injected search context, redo the arithmetic — the allowance and +the ration are both sized to a single measurement and should be re-measured +when the tool changes. If a search request is ever observed above 256k +input tokens in production, that is the signal to raise the allowance +rather than to quietly accept the overshoot. diff --git a/gateway/DESIGN.md b/gateway/DESIGN.md index 7f0207f..73725ae 100644 --- a/gateway/DESIGN.md +++ b/gateway/DESIGN.md @@ -300,6 +300,7 @@ dollar.** | `DSGATE_ANON_DAILY_REQUESTS` | 30 | enough to be useful for a day's work, not enough to script against | | `DSGATE_ANON_DAILY_INPUT_TOKENS` | 60000 | ~150 pages of context per day | | `DSGATE_ANON_DAILY_OUTPUT_TOKENS` | 20000 | the expensive side; the real cap | +| `DSGATE_ANON_DAILY_SEARCHES` | 3 | a `web_search` request costs ~10 ordinary turns; the request counter alone would let one caller take a quarter of the day | | `DSGATE_ANON_MAX_TOKENS` | 4096 | clamps a single response, bounding overshoot | | `DSGATE_MAX_BODY_BYTES` | 131072 | ~32K tokens; bounds the input side of overshoot | | `DSGATE_DAILY_BUDGET_USD` | 1.00 | the circuit breaker; the number that actually protects us | @@ -312,6 +313,19 @@ dollar.** | `DSGATE_TOKEN_TTL_DAYS` | 7 | identities age out instead of accumulating | | `DSGATE_BALANCE_CHECK_MINUTES` | 15 | the ledger's "we have credit" is checked against the real account | +**`web_search` is carried, and rationed.** Measured on 2026-08-07, one +search request made eleven server-side calls and billed 40,260 input +tokens with no separate per-search fee — so its whole cost arrives as +input tokens the meter already reads. What it breaks is the *reservation*, +which bounded input at one token per body byte: DeepSeek chooses how many +pages to read, so a search's input is upstream-controlled. Hence a 256k +input allowance at admission (about 6x the observed case) plus the daily +ration above. Within that allowance the budget is still a hard ceiling; +past it a search can overshoot by the difference, bounded by how many +distinct callers can be mid-search at once. Every other server-side tool +stays refused: unknown work at an unknown price, spent from donated +credit. Reasoning and the expiry condition are in `TASTE.md`. + **Free tier is flash only.** Pro is 3x the price and the request is *rejected*, not silently downgraded — a user who asked for pro and got flash without being told would draw wrong conclusions and blame the model. @@ -333,6 +347,8 @@ What that document says and what it refuses to say are both deliberate: | subject ids **truncated to 6 chars** | whole subject ids | a whole id can be matched against the one in someone's `free.json` | | **per-country** request counts | anything per-IP, ever | an aggregate is a fact about the service, not about a person | | token counts, tok/s, live subjects | prompts, completions | the promise in `deepseek free` is the promise here | +| **requests per day** for 30 days | per-day spend | a spend series is a map of when we are cheapest to empty | +| **upstream** last-success, fault streak | upstream key or account detail | "is it you or DeepSeek" is the question a failing caller has | Exact money, per-key health and the full subject table live behind `GET /admin/status` with the operator token. @@ -342,6 +358,24 @@ maps, everything in memory and lost on restart. That is the right trade — losing it costs a graph, and it keeps observability from ever becoming a second, weaker copy of the money. +**The main chart is a day per bar, not a second.** It was output tokens +per second over five minutes, which is the wrong instrument for a pool +serving a few requests an hour: idle almost every second, so the line was +flat at zero whenever anyone looked and said nothing true about whether +the service works. The daily series comes from `quota.History`, which +reads the journals — the same files the money settles from, so the pretty +chart cannot disagree with the ledger. Finished days are immutable and +memoised; only today is recomputed. Quiet days are present as zeroes, +because a gap in a chart reads as missing data rather than as a quiet day. + +**Upstream health is first-party, not scraped.** Every round trip to +DeepSeek is recorded with its outcome, and only *their* failures (429, 5xx, +transport) count as faults — a caller's own 4xx would otherwise read as an +outage. Three consecutive faults reads as `down`, one as `degraded`, and +before the first forwarded request the state is `unknown` rather than a +cheerful all-clear it has not earned. `status.deepseek.com` is deliberately +not consumed; see `TASTE.md` for the probe results. + ## The key pool One key was a single point of failure with a hard floor. Package diff --git a/gateway/cmd/dsgate/main.go b/gateway/cmd/dsgate/main.go index 4656e32..7cb559b 100644 --- a/gateway/cmd/dsgate/main.go +++ b/gateway/cmd/dsgate/main.go @@ -74,6 +74,7 @@ Per-user daily limits: DSGATE_ANON_DAILY_REQUESTS (30) DSGATE_ANON_DAILY_INPUT_TOKENS (60000) DSGATE_ANON_DAILY_OUTPUT_TOKENS (20000) + DSGATE_ANON_DAILY_SEARCHES (3) server-side web searches per user DSGATE_ANON_MAX_TOKENS (4096) per-request output cap DSGATE_MAX_BODY_BYTES (131072) per-request body cap DSGATE_REQUESTS_PER_MINUTE (20) per-address burst @@ -132,6 +133,7 @@ func run() error { DailyRequests: envInt("DSGATE_ANON_DAILY_REQUESTS", 30), DailyInputTokens: envInt("DSGATE_ANON_DAILY_INPUT_TOKENS", 60000), DailyOutputTokens: envInt("DSGATE_ANON_DAILY_OUTPUT_TOKENS", 20000), + DailySearches: envInt("DSGATE_ANON_DAILY_SEARCHES", 3), DailyBudgetUSD: envFloat("DSGATE_DAILY_BUDGET_USD", 1.00), TotalBudgetUSD: envFloat("DSGATE_TOTAL_BUDGET_USD", 20.00), } diff --git a/gateway/internal/meter/meter.go b/gateway/internal/meter/meter.go index 02aa17e..8cd7393 100644 --- a/gateway/internal/meter/meter.go +++ b/gateway/internal/meter/meter.go @@ -89,9 +89,17 @@ func Cost(model string, u Usage) float64 { // default upstream, and DeepSeek does not document that reasoning // tokens respect max_tokens — they are billed as output either way, // so the bound assumes they do not. -func Estimate(model string, requestBytes, maxTokens int) float64 { +// +// A search request breaks the first rule: the pages DeepSeek reads on the +// caller's behalf arrive as input tokens the body never contained, so +// searchInputAllowance is added to the input bound instead. +func Estimate(model string, requestBytes, maxTokens int, search bool) float64 { + input := requestBytes + 1 + if search { + input += searchInputAllowance + } return Cost(model, Usage{ - InputTokens: requestBytes + 1, + InputTokens: input, OutputTokens: maxTokens + reasoningAllowance, Found: false, }) @@ -103,6 +111,24 @@ func Estimate(model string, requestBytes, maxTokens int) float64 { // a cent, so over-reserving costs headroom, not money. const reasoningAllowance = 32 << 10 +// searchInputAllowance is the input headroom reserved for a server-side +// web search, whose page reads land in input_tokens without ever passing +// through the request body. +// +// 256k is a judgement, not a proof. A search request measured live on +// 2026-08-07 reported 40,260 input tokens after eleven server-side calls, +// so this is roughly six times the observed case; the model's 1M context +// is the only true bound, and reserving 1M would price a single search at +// more than half a day's budget and make the feature unofferable. +// +// The honest statement of the trade: within this allowance the budget is +// still a hard ceiling, and beyond it a search request can overshoot by +// the difference. Two things keep that survivable — the per-subject +// in-flight cap means one caller cannot stack such requests, and searches +// are rationed per user per day, so the overshoot is bounded by the few +// distinct callers who can be mid-search at the same moment. +const searchInputAllowance = 256 << 10 + // rawUsage is permissive on purpose: it decodes the usage object of every // format at once, using pointers so "absent" and "zero" stay distinct. // Which fields are present is what identifies the format. diff --git a/gateway/internal/meter/meter_test.go b/gateway/internal/meter/meter_test.go index f5f6bef..f778425 100644 --- a/gateway/internal/meter/meter_test.go +++ b/gateway/internal/meter/meter_test.go @@ -176,7 +176,7 @@ func TestEstimateExceedsATypicalRealCharge(t *testing.T) { const body = 4000 const maxTokens = 4096 - est := Estimate("deepseek-v4-flash", body, maxTokens) + est := Estimate("deepseek-v4-flash", body, maxTokens, false) real := Cost("deepseek-v4-flash", Usage{InputTokens: body / 3, OutputTokens: 800, Found: true}) if est <= real { t.Errorf("estimate %v is not above a realistic charge %v; unbillable would be cheaper than billable", est, real) @@ -218,3 +218,23 @@ func itoa(n int) string { } return string(b) } + +// A search request's input is chosen by DeepSeek, not by the caller: the +// pages it reads are billed as input tokens that never passed through the +// body. So the reservation cannot be derived from the body alone, and a +// search must hold materially more than the same bytes without one. +func TestSearchReservesBeyondTheBody(t *testing.T) { + const model = "deepseek-v4-flash" + plain := Estimate(model, 400, 1000, false) + search := Estimate(model, 400, 1000, true) + + if search <= plain { + t.Fatalf("a search reserved %v, no more than the %v an ordinary request holds", search, plain) + } + // The measured case on 2026-08-07 was 40k input tokens; the reservation + // has to cover that with room, or the ceiling leaks on every search. + measured := Cost(model, Usage{InputTokens: 40_260, CacheHitTokens: 32_000, OutputTokens: 3_100}) + if search < measured { + t.Errorf("reservation %v is under the %v a real measured search cost", search, measured) + } +} diff --git a/gateway/internal/policy/policy.go b/gateway/internal/policy/policy.go index 67484ab..ffaedc1 100644 --- a/gateway/internal/policy/policy.go +++ b/gateway/internal/policy/policy.go @@ -88,6 +88,12 @@ type Decision struct { // estimate if the response turns out to be unmeterable. MaxTokens int Stream bool + // Search is set when the request asks for DeepSeek's server-side web + // search. It travels because such a request costs a multiple of an + // ordinary one: the server injects the pages it read as input tokens, + // so neither the body's size nor MaxTokens predicts the bill. The + // reservation and the per-user ration both key off this. + Search bool } // Reject is a request refused before it cost anything. @@ -131,7 +137,7 @@ func Apply(route Route, body []byte, subject string, lim Limits) (*Decision, err if err := forbidFanOut(obj); err != nil { return nil, err } - if err := forbidServerTools(obj, route.Format); err != nil { + if err := checkServerTools(obj, route.Format, d); err != nil { return nil, err } setIdentity(obj, route.Format, subject) @@ -233,12 +239,29 @@ func forbidFanOut(obj map[string]any) error { return nil } -// forbidServerTools refuses tools that run on DeepSeek's side. Client -// tools ("function") only declare a schema and cost nothing extra; a -// server-side tool like web_search performs billed work that never -// appears in the usage object, which would put its cost outside every -// ceiling this gateway enforces. Only the Responses format offers them. -func forbidServerTools(obj map[string]any, f Format) error { +// checkServerTools decides which tools that run on DeepSeek's side the +// free tier will carry. Client tools ("function") only declare a schema +// and cost nothing extra. Only the Responses format offers server-side +// ones at all. +// +// web_search is allowed, and the reason is a measurement rather than a +// guess. Against the live API on 2026-08-07, one search request made 11 +// server-side calls (searches, page opens, an in-page find) and reported +// 40,260 input tokens, 32,000 of them cache hits — and the account +// balance moved by nothing beyond those tokens. So DeepSeek charges no +// per-search fee: the whole cost of a search arrives as input tokens in +// the usage object, which is exactly what this gateway already meters. +// Eleven searches at a frontier vendor's $10-per-1,000 rate would have +// been $0.11 and unmistakable in the balance; it was not there. +// +// What that measurement does change is the reservation. A search +// request's input is chosen by the server, not by the caller, so the +// request body no longer bounds it — see meter.Estimate. +// +// Every other server-side tool stays refused: an unknown tool is unknown +// work at an unknown price, and the honest default for spending someone +// else's donated credit is no. +func checkServerTools(obj map[string]any, f Format, d *Decision) error { if f != FormatResponses { return nil } @@ -246,16 +269,26 @@ func forbidServerTools(obj map[string]any, f Format) error { for _, t := range tools { tool, _ := t.(map[string]any) kind, _ := tool["type"].(string) - if kind != "" && kind != "function" { + switch { + case kind == "" || kind == "function": + case isWebSearch(kind): + d.Search = true + default: return &Reject{ Message: fmt.Sprintf("the free tier does not serve server-side tools (%q)", kind), - Hint: "bring your own key for web search: https://platform.deepseek.com/api_keys", + Hint: "web_search works here; for anything else bring your own key: https://platform.deepseek.com/api_keys", } } } return nil } +// isWebSearch matches the tool DeepSeek documents under two names, the +// bare one and the dated one their Responses API also accepts. +func isWebSearch(kind string) bool { + return kind == "web_search" || strings.HasPrefix(kind, "web_search_") +} + // setIdentity stamps the subject onto the request in whichever field the // format actually reads. // diff --git a/gateway/internal/policy/policy_test.go b/gateway/internal/policy/policy_test.go index 1e125c5..8671657 100644 --- a/gateway/internal/policy/policy_test.go +++ b/gateway/internal/policy/policy_test.go @@ -275,20 +275,39 @@ func asReject(err error, target **Reject) bool { return ok } -// Server-side tools perform billed work that never appears in the usage -// object — outside every ceiling this gateway enforces. Client function -// tools only declare a schema and stay allowed. +// web_search is carried, because measurement showed its whole cost +// arrives as input tokens this gateway already meters. Every other +// server-side tool is unknown work at an unknown price and stays refused. +// Client function tools only declare a schema and were never in question. func TestServerSideToolsAreRefused(t *testing.T) { route, _ := Lookup("POST", "/responses") lim := Limits{MaxTokens: 100, Model: "deepseek-v4-flash"} - _, err := Apply(route, []byte(`{"input":"hi","tools":[{"type":"web_search"}]}`), "sub", lim) - if err == nil { - t.Fatal("web_search passed policy; its per-search cost has no ceiling") + for _, kind := range []string{"web_search", "web_search_2025_08_26"} { + d, err := Apply(route, []byte(`{"input":"hi","tools":[{"type":"`+kind+`"}]}`), "sub", lim) + if err != nil { + t.Fatalf("%s was refused: %v", kind, err) + } + if !d.Search { + t.Errorf("%s did not set Decision.Search, so it would be reserved and rationed as an ordinary request", kind) + } + } + + // An unknown server-side tool is still a refusal, and the message has + // to point at the one that does work rather than only at the exit. + var rej *Reject + _, err := Apply(route, []byte(`{"input":"hi","tools":[{"type":"code_interpreter"}]}`), "sub", lim) + if !asReject(err, &rej) { + t.Fatalf("an unknown server-side tool passed policy: %v", err) + } + if !strings.Contains(rej.Hint, "web_search") { + t.Errorf("the refusal does not mention the tool that works: %q", rej.Hint) } - if _, err := Apply(route, []byte(`{"input":"hi","tools":[{"type":"function","name":"f"}]}`), "sub", lim); err != nil { + if d, err := Apply(route, []byte(`{"input":"hi","tools":[{"type":"function","name":"f"}]}`), "sub", lim); err != nil { t.Errorf("a client function tool was refused: %v", err) + } else if d.Search { + t.Error("a function tool was counted as a search") } // The other formats have no server-side tools; their tools stay open. diff --git a/gateway/internal/quota/lifetime_test.go b/gateway/internal/quota/lifetime_test.go index 9c8494c..c0db0f4 100644 --- a/gateway/internal/quota/lifetime_test.go +++ b/gateway/internal/quota/lifetime_test.go @@ -171,7 +171,7 @@ func TestCreditPoolStaysExhaustedAcrossARestart(t *testing.T) { l, done := open(t, dir, poolLimits()) defer done() - err := l.Admit("a-brand-new-subject", 0) + err := l.Admit("a-brand-new-subject", Admission{}) if err == nil { t.Fatal("admitted a request with the credit pool already empty") } diff --git a/gateway/internal/quota/quota.go b/gateway/internal/quota/quota.go index dfe41e3..bff8c6d 100644 --- a/gateway/internal/quota/quota.go +++ b/gateway/internal/quota/quota.go @@ -31,6 +31,7 @@ type Limits struct { DailyRequests int DailyInputTokens int DailyOutputTokens int + DailySearches int // DailyBudgetUSD is the circuit breaker: the total this service may // spend across all users in one UTC day. This is the number that @@ -48,6 +49,7 @@ type Account struct { Requests int `json:"requests"` InputTokens int `json:"input_tokens"` OutputTokens int `json:"output_tokens"` + Searches int `json:"searches"` SpentUSD float64 `json:"spent_usd"` } @@ -71,6 +73,22 @@ type UserCaps struct { Requests int `json:"requests"` InputTokens int `json:"input_tokens"` OutputTokens int `json:"output_tokens"` + // Searches rations requests that use DeepSeek's server-side web + // search. It exists because such a request costs roughly ten times an + // ordinary turn — the pages it reads are billed as input tokens — so + // the request count alone would let one caller take a large share of + // the day's budget while looking like a normal user. + Searches int `json:"searches"` +} + +// Admission is what a request asks the ledger for before it is forwarded. +// It is a struct rather than another positional argument because the two +// fields answer different questions — how much money to hold, and which +// per-user ration to spend — and a bare `true` at a call site would say +// neither. +type Admission struct { + ReserveUSD float64 + Search bool } // Reason classifies a refusal so the HTTP layer can pick a status code @@ -81,6 +99,7 @@ const ( ReasonRequests Reason = "daily_requests" ReasonInputTokens Reason = "daily_input_tokens" ReasonOutputTokens Reason = "daily_output_tokens" + ReasonSearches Reason = "daily_searches" ReasonDailyBudget Reason = "daily_budget" ReasonCredits Reason = "credits_exhausted" ReasonRevoked Reason = "revoked" @@ -109,6 +128,14 @@ func (e *LimitError) Error() string { return "this token has been revoked" case ReasonUnavailable: return "the free tier cannot record spend right now" + case ReasonRequests: + return "you have used today's request allowance" + case ReasonInputTokens: + return "you have used today's input-token allowance" + case ReasonOutputTokens: + return "you have used today's output-token allowance" + case ReasonSearches: + return "you have used today's web-search allowance" default: return fmt.Sprintf("daily %s limit reached", string(e.Reason)) } @@ -162,6 +189,12 @@ type Ledger struct { journalErr error now func() time.Time + + // history memoises finished days for the dashboard's daily series. It + // has its own lock because it is read on a status request and must not + // queue behind a request being admitted. + histMu sync.Mutex + history map[string]Day } // entry is one line of the journal. @@ -492,7 +525,8 @@ func (l *Ledger) reopenLocked() { // admitted only if, priced at its absolute maximum, it still fits under // both. Charge and Refund release the reservation, so the actual (almost // always much smaller) cost is what sticks. -func (l *Ledger) Admit(subject string, reserveUSD float64) error { +func (l *Ledger) Admit(subject string, req Admission) error { + reserveUSD := req.ReserveUSD l.mu.Lock() defer l.mu.Unlock() l.rollLocked() @@ -530,9 +564,17 @@ func (l *Ledger) Admit(subject string, reserveUSD float64) error { return &LimitError{Reason: ReasonInputTokens, ResetsAt: reset} case a.OutputTokens >= l.limits.DailyOutputTokens: return &LimitError{Reason: ReasonOutputTokens, ResetsAt: reset} + case req.Search && a.Searches >= l.limits.DailySearches: + return &LimitError{Reason: ReasonSearches, ResetsAt: reset} } a.Requests++ + if req.Search { + // Counted at admission rather than at settlement, because the + // ration has to bind before the money is spent: a search that + // failed still cost us the pages DeepSeek read. + a.Searches++ + } l.reserved += reserveUSD return nil } @@ -543,13 +585,18 @@ func (l *Ledger) Admit(subject string, reserveUSD float64) error { // Only failures the caller cannot provoke qualify — transport errors and // upstream 429/5xx. Refunding on anything the client controls, such as a // malformed body, would turn the request counter into a free retry loop. -func (l *Ledger) Refund(subject string, reserveUSD float64) { +func (l *Ledger) Refund(subject string, req Admission) { l.mu.Lock() defer l.mu.Unlock() - if a, ok := l.accounts[subject]; ok && a.Requests > 0 { - a.Requests-- + if a, ok := l.accounts[subject]; ok { + if a.Requests > 0 { + a.Requests-- + } + if req.Search && a.Searches > 0 { + a.Searches-- + } } - l.releaseLocked(reserveUSD) + l.releaseLocked(req.ReserveUSD) } // Release gives back a reservation while keeping the request debit, for @@ -629,6 +676,7 @@ func (l *Ledger) Status(subject, tier string) Status { Requests: l.limits.DailyRequests, InputTokens: l.limits.DailyInputTokens, OutputTokens: l.limits.DailyOutputTokens, + Searches: l.limits.DailySearches, }, ResetsAt: midnight(l.now()), Exhausted: l.priorSpend+l.daySpend >= l.limits.TotalBudgetUSD, @@ -794,3 +842,95 @@ func midnight(t time.Time) time.Time { u := t.UTC() return time.Date(u.Year(), u.Month(), u.Day(), 0, 0, 0, 0, time.UTC).Add(24 * time.Hour) } + +// Day is one UTC day of traffic, for the dashboard's history. +// +// No money, deliberately, for the same reason the public status document +// withholds it: a per-day spend series is a map of how much it takes to +// empty this service and when it is cheapest to try. +type Day struct { + Date string `json:"date"` + Requests int `json:"requests"` + InputTokens int `json:"input_tokens"` + OutputTokens int `json:"output_tokens"` + // Subjects is how many distinct anonymous identities sent something + // that day — the closest honest thing to "people", since no account + // exists. Only the count is published, never the ids. + Subjects int `json:"subjects"` +} + +// History is the last n UTC days, oldest first, including today. +// +// It is read from the journals rather than from a new counter, because the +// journals are already the record the money is settled from — a separate +// series could disagree with the ledger, and then the pretty chart would +// be the one people believe. A day that never had traffic is present with +// zeroes, so the series is a calendar rather than a list of events and the +// gaps are visible. +// +// Past days are immutable once the date rolls, so each is scanned at most +// once per process; only today is recomputed, from the live counters. +func (l *Ledger) History(days int) []Day { + if days < 1 { + days = 1 + } + l.mu.Lock() + l.rollLocked() + today := l.day + live := Day{Date: today, Subjects: len(l.accounts)} + for _, a := range l.accounts { + live.Requests += a.Requests + live.InputTokens += a.InputTokens + live.OutputTokens += a.OutputTokens + } + end, err := time.Parse("2006-01-02", today) + l.mu.Unlock() + if err != nil { + return []Day{live} + } + + out := make([]Day, 0, days) + for i := days - 1; i >= 0; i-- { + date := end.AddDate(0, 0, -i).Format("2006-01-02") + if date == today { + out = append(out, live) + continue + } + out = append(out, l.pastDay(date)) + } + return out +} + +// pastDay reads one finished day out of its journal, remembering the +// answer: a day that has ended cannot change. +func (l *Ledger) pastDay(date string) Day { + l.histMu.Lock() + if d, ok := l.history[date]; ok { + l.histMu.Unlock() + return d + } + l.histMu.Unlock() + + d := Day{Date: date} + if f, err := os.Open(l.journalPath(date)); err == nil { + seen := make(map[string]struct{}) + scanJournal(f, func(e entry) { + d.Requests++ + d.InputTokens += e.InputTokens + d.OutputTokens += e.OutputTokens + if e.Subject != "" { + seen[e.Subject] = struct{}{} + } + }) + f.Close() + d.Subjects = len(seen) + } + + l.histMu.Lock() + if l.history == nil { + l.history = make(map[string]Day) + } + l.history[date] = d + l.histMu.Unlock() + return d +} diff --git a/gateway/internal/quota/quota_test.go b/gateway/internal/quota/quota_test.go index 1d3e368..68cafaf 100644 --- a/gateway/internal/quota/quota_test.go +++ b/gateway/internal/quota/quota_test.go @@ -42,11 +42,11 @@ func TestRequestsAreCappedPerDay(t *testing.T) { defer done() for i := 0; i < 3; i++ { - if err := l.Admit("alice", 0); err != nil { + if err := l.Admit("alice", Admission{}); err != nil { t.Fatalf("request %d refused: %v", i+1, err) } } - err := l.Admit("alice", 0) + err := l.Admit("alice", Admission{}) if err == nil { t.Fatal("a fourth request was admitted against a limit of three") } @@ -55,7 +55,7 @@ func TestRequestsAreCappedPerDay(t *testing.T) { } // One subject hitting its limit must not affect another. - if err := l.Admit("bob", 0); err != nil { + if err := l.Admit("bob", Admission{}); err != nil { t.Errorf("bob was refused because alice ran out: %v", err) } } @@ -66,12 +66,12 @@ func TestTokenLimitsAreEnforcedAfterCharging(t *testing.T) { l, done := open(t, t.TempDir(), lim) defer done() - if err := l.Admit("alice", 0); err != nil { + if err := l.Admit("alice", Admission{}); err != nil { t.Fatal(err) } l.Charge("alice", "chat", "deepseek-v4-flash", 0, 0, 600, 0.0001, 0, false) - err := l.Admit("alice", 0) + err := l.Admit("alice", Admission{}) if err == nil { t.Fatal("admitted after the output token cap was passed") } @@ -94,7 +94,7 @@ func TestDailyBudgetStopsEveryone(t *testing.T) { spenders := []string{"a", "b", "c", "d", "e"} tripped := "" for _, who := range spenders { - if err := l.Admit(who, 0); err != nil { + if err := l.Admit(who, Admission{}); err != nil { tripped = who break } @@ -106,7 +106,7 @@ func TestDailyBudgetStopsEveryone(t *testing.T) { // And it holds for someone who has spent nothing at all: the breaker // is about the service's money, not the caller's behaviour. - err := l.Admit("someone-brand-new", 0) + err := l.Admit("someone-brand-new", Admission{}) if err == nil { t.Fatal("a fresh subject was admitted after the daily budget was spent") } @@ -122,10 +122,10 @@ func TestCreditsExhaustedOutranksTheDailyBudget(t *testing.T) { l, done := open(t, t.TempDir(), lim) defer done() - l.Admit("a", 0) + l.Admit("a", Admission{}) l.Charge("a", "chat", "deepseek-v4-flash", 0, 0, 0, 0.10, 0, false) - err := l.Admit("b", 0) + err := l.Admit("b", Admission{}) if got := reasonOf(t, err); got != ReasonCredits { t.Fatalf("reason = %q, want %q", got, ReasonCredits) } @@ -144,9 +144,9 @@ func TestCountersSurviveARestart(t *testing.T) { dir := t.TempDir() l, _ := open(t, dir, testLimits()) - l.Admit("alice", 0) + l.Admit("alice", Admission{}) l.Charge("alice", "chat", "deepseek-v4-flash", 400, 0, 200, 0.002, 0, false) - l.Admit("alice", 0) + l.Admit("alice", Admission{}) l.Charge("alice", "chat", "deepseek-v4-flash", 100, 0, 50, 0.001, 0, false) l.Close() @@ -170,7 +170,7 @@ func TestCountersSurviveARestart(t *testing.T) { func TestTruncatedJournalKeepsWhatItCan(t *testing.T) { dir := t.TempDir() l, _ := open(t, dir, testLimits()) - l.Admit("alice", 0) + l.Admit("alice", Admission{}) l.Charge("alice", "chat", "deepseek-v4-flash", 400, 0, 200, 0.002, 0, false) day := l.day l.Close() @@ -199,7 +199,7 @@ func TestLifetimeSpendSurvivesTheDayRolling(t *testing.T) { l, _ := open(t, dir, testLimits()) l.SetClock(func() time.Time { return now }) - l.Admit("alice", 0) + l.Admit("alice", Admission{}) l.Charge("alice", "chat", "deepseek-v4-flash", 100, 0, 100, 0.05, 0, false) now = now.Add(2 * time.Minute) // past midnight UTC @@ -241,14 +241,14 @@ func TestRefundReturnsTheRequestAllowance(t *testing.T) { l, done := open(t, t.TempDir(), testLimits()) defer done() - l.Admit("alice", 0) - l.Refund("alice", 0) + l.Admit("alice", Admission{}) + l.Refund("alice", Admission{}) if st := l.Status("alice", "anon"); st.Used.Requests != 0 { t.Errorf("requests after refund = %d, want 0", st.Used.Requests) } // Refunding more than was taken must not create allowance. - l.Refund("alice", 0) - l.Refund("alice", 0) + l.Refund("alice", Admission{}) + l.Refund("alice", Admission{}) if st := l.Status("alice", "anon"); st.Used.Requests != 0 { t.Errorf("over-refunding produced %d requests", st.Used.Requests) } @@ -259,7 +259,7 @@ func TestRevocation(t *testing.T) { l, done := open(t, dir, testLimits()) defer done() - if err := l.Admit("spammer", 0); err != nil { + if err := l.Admit("spammer", Admission{}); err != nil { t.Fatal(err) } if err := os.WriteFile(filepath.Join(dir, "revoked.txt"), @@ -270,10 +270,10 @@ func TestRevocation(t *testing.T) { t.Fatal(err) } - if got := reasonOf(t, l.Admit("spammer", 0)); got != ReasonRevoked { + if got := reasonOf(t, l.Admit("spammer", Admission{})); got != ReasonRevoked { t.Errorf("reason = %q, want %q", got, ReasonRevoked) } - if err := l.Admit("alice", 0); err != nil { + if err := l.Admit("alice", Admission{}); err != nil { t.Errorf("revoking one subject blocked another: %v", err) } } @@ -284,7 +284,7 @@ func TestRevocation(t *testing.T) { func TestJournalRecordsCountsAndNothingElse(t *testing.T) { dir := t.TempDir() l, _ := open(t, dir, testLimits()) - l.Admit("alice", 0) + l.Admit("alice", Admission{}) l.Charge("alice", "chat", "deepseek-v4-flash", 400, 120, 200, 0.002, 0, false) day := l.day l.Close() @@ -309,7 +309,7 @@ func TestJournalRecordsCountsAndNothingElse(t *testing.T) { func TestStatusDoesNotLeakServiceFinances(t *testing.T) { l, done := open(t, t.TempDir(), testLimits()) defer done() - l.Admit("alice", 0) + l.Admit("alice", Admission{}) l.Charge("alice", "chat", "deepseek-v4-flash", 10, 0, 10, 0.005, 0, false) st := l.Status("alice", "anon") @@ -333,20 +333,20 @@ func TestReservationIsACeiling(t *testing.T) { l, done := open(t, t.TempDir(), lim) defer done() - if got := reasonOf(t, l.Admit("a", 0.02)); got != ReasonDailyBudget { + if got := reasonOf(t, l.Admit("a", Admission{ReserveUSD: 0.02})); got != ReasonDailyBudget { t.Fatalf("a $0.02 worst case fit under a $0.01 budget: %v", got) } - if err := l.Admit("a", 0.005); err != nil { + if err := l.Admit("a", Admission{ReserveUSD: 0.005}); err != nil { t.Fatalf("a fitting reservation was refused: %v", err) } // A second request that fits the budget alone, but not alongside the // first one's reservation, must wait. - if got := reasonOf(t, l.Admit("b", 0.006)); got != ReasonDailyBudget { + if got := reasonOf(t, l.Admit("b", Admission{ReserveUSD: 0.006})); got != ReasonDailyBudget { t.Fatalf("overlapping reservations overshot the budget: %v", got) } // Settling the first request at its (small) real cost frees the room. l.Charge("a", "chat", "deepseek-v4-flash", 10, 0, 10, 0.001, 0.005, false) - if err := l.Admit("b", 0.006); err != nil { + if err := l.Admit("b", Admission{ReserveUSD: 0.006}); err != nil { t.Fatalf("room was not released at Charge: %v", err) } } @@ -357,15 +357,15 @@ func TestRefundReleasesTheReservation(t *testing.T) { l, done := open(t, t.TempDir(), lim) defer done() - if err := l.Admit("a", 0.009); err != nil { + if err := l.Admit("a", Admission{ReserveUSD: 0.009}); err != nil { t.Fatal(err) } - l.Refund("a", 0.009) - if err := l.Admit("b", 0.009); err != nil { + l.Refund("a", Admission{ReserveUSD: 0.009}) + if err := l.Admit("b", Admission{ReserveUSD: 0.009}); err != nil { t.Fatalf("a refunded reservation still held the budget: %v", err) } l.Release(0.009) - if err := l.Admit("c", 0.009); err != nil { + if err := l.Admit("c", Admission{ReserveUSD: 0.009}); err != nil { t.Fatalf("Release did not free the room: %v", err) } } @@ -376,7 +376,7 @@ func TestJournalFailureFailsClosed(t *testing.T) { l, done := open(t, t.TempDir(), testLimits()) defer done() - if err := l.Admit("a", 0); err != nil { + if err := l.Admit("a", Admission{}); err != nil { t.Fatal(err) } @@ -391,7 +391,7 @@ func TestJournalFailureFailsClosed(t *testing.T) { } l.Charge("a", "chat", "deepseek-v4-flash", 10, 0, 10, 0.0001, 0, false) - err := l.Admit("b", 0) + err := l.Admit("b", Admission{}) if got := reasonOf(t, err); got != ReasonUnavailable { t.Fatalf("admissions continued with a dead journal: %v", err) } @@ -400,7 +400,7 @@ func TestJournalFailureFailsClosed(t *testing.T) { if err := os.Chmod(path, 0o600); err != nil { t.Fatal(err) } - if err := l.Admit("b", 0); err != nil { + if err := l.Admit("b", Admission{}); err != nil { t.Fatalf("the ledger did not recover after the journal came back: %v", err) } } @@ -410,7 +410,7 @@ func TestJournalFailureFailsClosed(t *testing.T) { func TestReplaySkipsACorruptLine(t *testing.T) { dir := t.TempDir() l, done := open(t, dir, testLimits()) - l.Admit("a", 0) + l.Admit("a", Admission{}) l.Charge("a", "chat", "deepseek-v4-flash", 10, 0, 10, 0.002, 0, false) day := l.day done() @@ -424,7 +424,7 @@ func TestReplaySkipsACorruptLine(t *testing.T) { f.Close() l2, done2 := open(t, dir, testLimits()) - l2.Admit("a", 0) + l2.Admit("a", Admission{}) l2.Charge("a", "chat", "deepseek-v4-flash", 10, 0, 10, 0.003, 0, false) done2() @@ -434,3 +434,51 @@ func TestReplaySkipsACorruptLine(t *testing.T) { t.Errorf("day spend replayed as $%.4f; the corrupt line ate the entries after it", got) } } + +// A search costs roughly ten times an ordinary turn, so it has its own +// ration. Two properties matter and neither is obvious: running out of +// searches must not touch the rest of the tier, and a search that never +// reached the model must give the ration back. +func TestSearchesAreRationedWithoutBlockingOrdinaryRequests(t *testing.T) { + lim := testLimits() + lim.DailyRequests = 100 + lim.DailySearches = 2 + l, done := open(t, t.TempDir(), lim) + defer done() + + for i := 0; i < 2; i++ { + if err := l.Admit("alice", Admission{Search: true}); err != nil { + t.Fatalf("search %d refused: %v", i+1, err) + } + } + + err := l.Admit("alice", Admission{Search: true}) + if err == nil { + t.Fatal("a third search was admitted against a ration of two") + } + if got := reasonOf(t, err); got != ReasonSearches { + t.Errorf("reason = %q, want %q", got, ReasonSearches) + } + + // The point of a separate ration: everything else still works. + if err := l.Admit("alice", Admission{}); err != nil { + t.Errorf("an ordinary request was refused because searches ran out: %v", err) + } + + // And a refunded search is not a spent one. + l.Refund("alice", Admission{Search: true}) + if err := l.Admit("alice", Admission{Search: true}); err != nil { + t.Errorf("a refunded search ration was not returned: %v", err) + } +} + +func TestSearchRationIsPublishedWithTheOtherLimits(t *testing.T) { + lim := testLimits() + lim.DailySearches = 3 + l, done := open(t, t.TempDir(), lim) + defer done() + + if got := l.Status("alice", "anon").Limits.Searches; got != 3 { + t.Errorf("published search limit = %d, want 3 — a limit a caller cannot read is one they can only discover by hitting it", got) + } +} diff --git a/gateway/internal/server/proxy.go b/gateway/internal/server/proxy.go index a715f5f..0727391 100644 --- a/gateway/internal/server/proxy.go +++ b/gateway/internal/server/proxy.go @@ -116,7 +116,7 @@ func (s *Server) handleProxy(w http.ResponseWriter, r *http.Request) { // `deepseek status` free and safe in a loop against the free tier, // exactly as it is against the real API. billable := route.Format != policy.FormatNone - var reserve float64 + adm := quota.Admission{Search: decision.Search} if billable { if s.upstreamDry.Load() { // DeepSeek says the account is unusable. The local ledger's @@ -129,8 +129,8 @@ func (s *Server) handleProxy(w http.ResponseWriter, r *http.Request) { // forwarded. This is what makes the budget a ceiling: without the // reservation, every in-flight request is unbilled and the breaker // only notices after the money is spent. - reserve = meter.Estimate(decision.Model, len(decision.Body), decision.MaxTokens) - if err := s.ledger.Admit(subject, reserve); err != nil { + adm.ReserveUSD = meter.Estimate(decision.Model, len(decision.Body), decision.MaxTokens, decision.Search) + if err := s.ledger.Admit(subject, adm); err != nil { s.writeLimit(w, err) return } @@ -141,7 +141,7 @@ func (s *Server) handleProxy(w http.ResponseWriter, r *http.Request) { if err := s.acquire(r); err != nil { if billable { - s.ledger.Refund(subject, reserve) + s.ledger.Refund(subject, adm) } w.Header().Set("Retry-After", "5") writeError(w, http.StatusServiceUnavailable, typeQuota, @@ -153,7 +153,7 @@ func (s *Server) handleProxy(w http.ResponseWriter, r *http.Request) { s.stats.InFlight(1) defer s.stats.InFlight(-1) - s.forward(w, r, route, decision, subject, billable, reserve) + s.forward(w, r, route, decision, subject, billable, adm) } // edgeCountry reads the two-letter country the CDN attached to this @@ -194,7 +194,8 @@ func (s *Server) acquire(r *http.Request) error { } } -func (s *Server) forward(w http.ResponseWriter, r *http.Request, route policy.Route, d *policy.Decision, subject string, billable bool, reserve float64) { +func (s *Server) forward(w http.ResponseWriter, r *http.Request, route policy.Route, d *policy.Decision, subject string, billable bool, adm quota.Admission) { + reserve := adm.ReserveUSD url := strings.TrimRight(s.cfg.UpstreamBaseURL, "/") + route.Upstream var payload io.Reader @@ -204,7 +205,7 @@ func (s *Server) forward(w http.ResponseWriter, r *http.Request, route policy.Ro up, err := http.NewRequestWithContext(r.Context(), route.Method, url, payload) if err != nil { if billable { - s.ledger.Refund(subject, reserve) + s.ledger.Refund(subject, adm) } writeError(w, http.StatusInternalServerError, typeInternal, "could not build the upstream request") return @@ -223,7 +224,7 @@ func (s *Server) forward(w http.ResponseWriter, r *http.Request, route policy.Ro secret, fingerprint, err := s.keys.Next() if err != nil { if billable { - s.ledger.Refund(subject, reserve) + s.ledger.Refund(subject, adm) } s.writeLimit(w, "a.LimitError{Reason: quota.ReasonCredits}) return @@ -239,6 +240,7 @@ func (s *Server) forward(w http.ResponseWriter, r *http.Request, route policy.Ro } up.Header.Set("User-Agent", "dsgate") + sent := time.Now() resp, err := s.http.Do(up) if err != nil { if r.Context().Err() != nil { @@ -258,13 +260,20 @@ func (s *Server) forward(w http.ResponseWriter, r *http.Request, route policy.Ro // Never reached the model, so it cost nothing and the caller keeps // their request allowance. if billable { - s.ledger.Refund(subject, reserve) + s.ledger.Refund(subject, adm) } + s.stats.Upstream("unreachable", true, 0) writeError(w, http.StatusBadGateway, typeUpstream, "could not reach DeepSeek: "+err.Error()) return } defer resp.Body.Close() + // One observation per round trip, and only DeepSeek's own failures + // count as faults: a 4xx is the caller's request coming back. + s.stats.Upstream(strconv.Itoa(resp.StatusCode), + resp.StatusCode == http.StatusTooManyRequests || resp.StatusCode >= 500, + time.Since(sent)) + // A key that upstream refuses for money or validity is done, and // leaves the rotation now rather than after it has failed everyone // else's request too. Other 4xx are about the request, not the key. @@ -326,7 +335,7 @@ func (s *Server) forward(w http.ResponseWriter, r *http.Request, route policy.Ro if upstreamFault { // No tokens were generated and the fault was not the // caller's: give everything back. - s.ledger.Refund(subject, reserve) + s.ledger.Refund(subject, adm) } else { // The caller's own 4xx keeps its request debit, but the // money reserved for it goes back to the pool. @@ -491,6 +500,12 @@ func (s *Server) writeLimit(w http.ResponseWriter, err error) { w.Header().Set("Retry-After", "30") writeError(w, http.StatusServiceUnavailable, typeInternal, "the free tier is temporarily unavailable; retry shortly") + case quota.ReasonSearches: + // A distinct message because the fix is distinct: the rest of the + // tier still works, so "come back tomorrow" would be wrong. + retryAfter(w, lim.RetryAfter(time.Now())) + writeError(w, http.StatusTooManyRequests, typeQuota, + "you have used today's web-search allowance. Ordinary requests still work — searches reset at 00:00 UTC, or bring your own key for unlimited search: https://platform.deepseek.com/api_keys") case quota.ReasonDailyBudget: retryAfter(w, lim.RetryAfter(time.Now())) writeError(w, http.StatusTooManyRequests, typeQuota, diff --git a/gateway/internal/server/server_test.go b/gateway/internal/server/server_test.go index 35e4790..1e8c585 100644 --- a/gateway/internal/server/server_test.go +++ b/gateway/internal/server/server_test.go @@ -89,6 +89,7 @@ func newHarness(t *testing.T, up *upstream, tune func(*Config, *quota.Limits)) * DailyRequests: 5, DailyInputTokens: 10000, DailyOutputTokens: 5000, + DailySearches: 2, DailyBudgetUSD: 1, TotalBudgetUSD: 10, } @@ -901,3 +902,55 @@ func TestUpstreamDryBalanceStopsAdmissions(t *testing.T) { t.Errorf("HTTP %d after the account recovered, want 200", resp2.StatusCode) } } + +// web_search has to work end to end on the free tier — it is the reason +// `deepseek respond --web-search` exists — and it has to stay rationed, +// because one search costs about what ten ordinary turns cost. +func TestWebSearchIsCarriedAndRationed(t *testing.T) { + up := newUpstream(t, func(w http.ResponseWriter, r *http.Request) { + io.WriteString(w, chatReply(100, 50)) + }) + h := newHarness(t, up, nil) // DailySearches: 2 + tok := h.enrol(t) + + const search = `{"input":"who won","tools":[{"type":"web_search"}]}` + for i := 0; i < 2; i++ { + resp := h.do(t, "POST", "/responses", tok, search) + resp.Body.Close() + if resp.StatusCode != http.StatusOK { + t.Fatalf("search %d: status = %d, want 200", i+1, resp.StatusCode) + } + h.settle(t) + } + // The tool must reach DeepSeek intact — a gateway that quietly dropped + // it would return a confidently unsourced answer. + tools, _ := up.last(t).Body["tools"].([]any) + if len(tools) != 1 { + t.Fatalf("upstream saw %d tools, want the one that was sent", len(tools)) + } + if kind, _ := tools[0].(map[string]any)["type"].(string); kind != "web_search" { + t.Errorf("upstream saw tool type %q, want web_search", kind) + } + + resp := h.do(t, "POST", "/responses", tok, search) + defer resp.Body.Close() + if resp.StatusCode != http.StatusTooManyRequests { + t.Fatalf("a third search past a ration of two: status = %d, want 429", resp.StatusCode) + } + var body struct { + Error struct { + Type string `json:"type"` + Message string `json:"message"` + } `json:"error"` + } + json.NewDecoder(resp.Body).Decode(&body) + if !strings.Contains(body.Error.Message, "search") { + t.Errorf("the refusal does not say searches ran out: %q", body.Error.Message) + } + // Ordinary requests must survive an exhausted search ration. + plain := h.do(t, "POST", "/responses", tok, `{"input":"hi"}`) + defer plain.Body.Close() + if plain.StatusCode != http.StatusOK { + t.Errorf("an ordinary request was refused after searches ran out: status = %d", plain.StatusCode) + } +} diff --git a/gateway/internal/server/status.go b/gateway/internal/server/status.go index 48828f7..47a955f 100644 --- a/gateway/internal/server/status.go +++ b/gateway/internal/server/status.go @@ -42,6 +42,16 @@ type PublicStatus struct { Countries []stats.Count `json:"countries"` Top []TopSubject `json:"top_subjects"` + // History is the last 30 UTC days of traffic, oldest first. A service + // this size serves a few requests an hour at best, which is why the + // live five-minute view is nearly always zero and cannot be the whole + // story — the daily series is where the traffic is actually visible. + History []quota.Day `json:"history"` + + // Upstream is what DeepSeek has done for us lately, so a visitor can + // tell our outage from theirs without leaving the page. + Upstream stats.Upstream `json:"upstream"` + Keys PoolStatus `json:"key_pool"` Limits quota.UserCaps `json:"daily_limits_per_user"` System stats.System `json:"system"` @@ -107,6 +117,11 @@ func withoutMoney(t quota.Totals) quota.Totals { return t } +// historyDays is how far the daily series goes back. Thirty days is one +// screen of bars at a readable width, and long enough that a week of +// growth or a quiet stretch is legible rather than a rounding error. +const historyDays = 30 + // publicStatusTTL caches the document. The dashboard polls, several // people may have it open, and every field is a five-minute rolling // figure — recomputing per request would spend more CPU on watching the @@ -171,6 +186,8 @@ func (s *Server) buildStatus() *PublicStatus { SubjectsToday: h.Subjects, }, Live: snap.Live, + History: s.ledger.History(historyDays), + Upstream: snap.Upstream, Endpoints: snap.Endpoints, Countries: snap.Countries, Top: rows, diff --git a/gateway/internal/server/status_test.go b/gateway/internal/server/status_test.go index 2cfe6f9..4dbc548 100644 --- a/gateway/internal/server/status_test.go +++ b/gateway/internal/server/status_test.go @@ -165,7 +165,7 @@ func TestStatusReportsExhaustion(t *testing.T) { // Spend the pool for real rather than configuring it to zero: the // state has to follow actual spend, which is the thing that goes // wrong in production. - h.ledger.Admit("someone", 0) + h.ledger.Admit("someone", quota.Admission{}) h.ledger.Charge("someone", "chat", "deepseek-v4-flash", 10, 0, 10, 0.002, 0, false) st := getStatus(t, h) @@ -339,3 +339,106 @@ func TestEmptyPoolIsAnHonest402(t *testing.T) { t.Errorf("state = %q, want %q", st.State, StateDry) } } + +// The daily series is the dashboard's main chart, so it has to be a +// calendar — every day present, quiet days as zeroes — and it has to keep +// the same secret the rest of the document keeps: no money. +func TestStatusHistoryIsACalendarWithoutMoney(t *testing.T) { + up := newUpstream(t, func(w http.ResponseWriter, r *http.Request) { + io.WriteString(w, chatReply(120, 80)) + }) + h := newHarness(t, up, nil) + tok := h.enrol(t) + h.do(t, "POST", "/chat/completions", tok, `{"messages":[]}`).Body.Close() + h.settle(t) + + resp := h.do(t, "GET", "/v1/status", "", "") + defer resp.Body.Close() + raw, _ := io.ReadAll(resp.Body) + + var st PublicStatus + if err := json.Unmarshal(raw, &st); err != nil { + t.Fatal(err) + } + if len(st.History) != historyDays { + t.Fatalf("history has %d days, want %d — gaps in a chart read as zero traffic, not as missing data", len(st.History), historyDays) + } + last := st.History[len(st.History)-1] + if last.Requests != 1 { + t.Errorf("today's row shows %d requests, want the 1 just served", last.Requests) + } + if last.Subjects != 1 { + t.Errorf("today's row shows %d subjects, want 1", last.Subjects) + } + // Dates must be ordered oldest first, or the chart draws backwards. + for i := 1; i < len(st.History); i++ { + if st.History[i-1].Date >= st.History[i].Date { + t.Fatalf("history is not oldest-first at %d: %q then %q", i, st.History[i-1].Date, st.History[i].Date) + } + } + // A per-day spend series would be a map of when we are cheapest to + // empty. It must not be in the public document at any nesting. + var probe struct { + History []map[string]any `json:"history"` + } + json.Unmarshal(raw, &probe) + for _, day := range probe.History { + for k := range day { + if strings.Contains(k, "usd") || strings.Contains(k, "spend") || strings.Contains(k, "cost") { + t.Errorf("the daily series carries money: %q", k) + } + } + } +} + +// "Is it you or DeepSeek" is the question a visitor actually has when a +// request fails, so the answer has to be on the page — and it must not +// claim health it has not observed. +func TestStatusReportsUpstreamHealth(t *testing.T) { + var fail bool + up := newUpstream(t, func(w http.ResponseWriter, r *http.Request) { + if fail { + w.WriteHeader(http.StatusBadGateway) + return + } + io.WriteString(w, chatReply(10, 10)) + }) + h := newHarness(t, up, nil) + tok := h.enrol(t) + + // Before anything has been asked of DeepSeek, the honest answer is + // "unknown" rather than a cheerful all-clear. + var st PublicStatus + resp := h.do(t, "GET", "/v1/status", "", "") + json.NewDecoder(resp.Body).Decode(&st) + resp.Body.Close() + if st.Upstream.State != "unknown" { + t.Errorf("upstream state before any call = %q, want unknown", st.Upstream.State) + } + + h.do(t, "POST", "/chat/completions", tok, `{"messages":[]}`).Body.Close() + h.settle(t) + h.invalidateStatus() + resp = h.do(t, "GET", "/v1/status", "", "") + json.NewDecoder(resp.Body).Decode(&st) + resp.Body.Close() + if st.Upstream.State != "ok" || st.Upstream.Calls != 1 { + t.Errorf("after one good call: state=%q calls=%d, want ok/1", st.Upstream.State, st.Upstream.Calls) + } + + fail = true + for i := 0; i < 3; i++ { + h.do(t, "POST", "/chat/completions", tok, `{"messages":[]}`).Body.Close() + h.settle(t) + } + h.invalidateStatus() + resp = h.do(t, "GET", "/v1/status", "", "") + json.NewDecoder(resp.Body).Decode(&st) + resp.Body.Close() + if st.Upstream.State != "down" { + t.Errorf("after three consecutive 502s: state=%q, want down", st.Upstream.State) + } + if st.Upstream.LastFault != "502" { + t.Errorf("last fault = %q, want 502", st.Upstream.LastFault) + } +} diff --git a/gateway/internal/server/web/app.js b/gateway/internal/server/web/app.js index 119a9c0..c63a6e1 100644 --- a/gateway/internal/server/web/app.js +++ b/gateway/internal/server/web/app.js @@ -107,23 +107,30 @@ pill.className = "pill " + cls; } - /* ---------- sparkline chart ---------- */ + /* ---------- daily chart ---------- + + Bars, one per UTC day, for the last thirty days. It used to be output + tokens per second over five minutes, which was the wrong instrument + for this service: a shared pool serving a few requests an hour is + idle almost every second, so the line was flat at zero whenever + anyone looked and said nothing true about whether the thing works. + A day is the smallest bucket that is usually non-empty here. */ var chart = (function () { var canvas = $("spark"); var ctx = canvas.getContext("2d"); - var target = []; // latest series from the server - var shown = []; // what is currently drawn (tweens toward target) + var days = []; // [{date, requests, input_tokens, output_tokens, subjects}] + var shown = []; // bar heights currently drawn (tween toward days) var tweenFrom = null; var tweenStart = 0; var raf = 0; var hoverIdx = -1; - var colors = { line: "#00c2e9", grid: "#9a9a9a" }; + var colors = { bar: "#00c2e9", grid: "#9a9a9a" }; var TWEEN_MS = 280; function refreshColors() { var cs = getComputedStyle(canvas); - colors.line = cs.color; + colors.bar = cs.color; colors.grid = cs.borderTopColor || cs.borderColor || colors.grid; } @@ -142,13 +149,11 @@ function draw() { var dim = size(); var w = dim.w, h = dim.h; - var pad = 4; + var pad = 6; ctx.clearRect(0, 0, w, h); - var s = shown; - var n = s.length; - - // baseline + // baseline: drawn even with no data, so an empty chart still reads + // as an axis rather than as a failed render. ctx.globalAlpha = 0.5; ctx.strokeStyle = colors.grid; ctx.lineWidth = 1; @@ -158,40 +163,32 @@ ctx.stroke(); ctx.globalAlpha = 1; - if (n < 2) return; + var n = shown.length; + if (n === 0) return; var max = 1; - for (var i = 0; i < n; i++) if (s[i] > max) max = s[i]; - - function x(i) { return (i / (n - 1)) * w; } - function y(v) { return h - pad - (v / max) * (h - pad * 2); } - - // area fill - ctx.beginPath(); - ctx.moveTo(0, h); - for (var j = 0; j < n; j++) ctx.lineTo(x(j), y(s[j])); - ctx.lineTo(w, h); - ctx.closePath(); - ctx.globalAlpha = 0.16; - ctx.fillStyle = colors.line; - ctx.fill(); - ctx.globalAlpha = 1; - - // line - ctx.beginPath(); - for (var k = 0; k < n; k++) { - if (k === 0) ctx.moveTo(x(k), y(s[k])); - else ctx.lineTo(x(k), y(s[k])); + for (var i = 0; i < n; i++) if (shown[i] > max) max = shown[i]; + + var slot = w / n; + var bw = Math.max(2, Math.min(slot - 2, 18)); + + for (var j = 0; j < n; j++) { + var cx = slot * (j + 0.5); + var v = shown[j]; + var bh = (v / max) * (h - pad - 1); + var isToday = j === n - 1; + // A zero day still gets a one-pixel tick. Nothing drawn at all + // looks like missing data; a floor line reads as a quiet day. + if (bh < 1) bh = v > 0 ? 1.5 : 1; + ctx.globalAlpha = v > 0 ? (isToday ? 1 : 0.75) : 0.22; + ctx.fillStyle = colors.bar; + ctx.fillRect(Math.round(cx - bw / 2), h - 1 - bh, Math.round(bw), bh); } - ctx.strokeStyle = colors.line; - ctx.lineWidth = 2; - ctx.lineJoin = "round"; - ctx.stroke(); + ctx.globalAlpha = 1; - // hover crosshair if (hoverIdx >= 0 && hoverIdx < n) { - var hx = x(hoverIdx); - ctx.globalAlpha = 0.6; + var hx = slot * (hoverIdx + 0.5); + ctx.globalAlpha = 0.5; ctx.strokeStyle = colors.grid; ctx.lineWidth = 1; ctx.beginPath(); @@ -199,41 +196,57 @@ ctx.lineTo(hx + 0.5, h); ctx.stroke(); ctx.globalAlpha = 1; - ctx.beginPath(); - ctx.arc(hx, y(s[hoverIdx]), 3.5, 0, Math.PI * 2); - ctx.fillStyle = colors.line; - ctx.fill(); } } function step(ts) { var t = Math.min(1, (ts - tweenStart) / TWEEN_MS); var e = 1 - Math.pow(1 - t, 3); // ease-out cubic - for (var i = 0; i < target.length; i++) { + for (var i = 0; i < days.length; i++) { var from = tweenFrom[i] || 0; - shown[i] = from + (target[i] - from) * e; + shown[i] = from + (days[i].requests - from) * e; } - shown.length = target.length; + shown.length = days.length; draw(); if (t < 1) raf = requestAnimationFrame(step); } - function update(series) { - if (!Array.isArray(series)) series = []; + function readout(i) { + var d = days[i]; + if (!d) return ""; + var when = d.date; + if (i === days.length - 1) when += " (today)"; + return when + " · " + fmtInt(d.requests) + " req · " + + fmtCompact(d.output_tokens) + " out · " + fmtInt(d.subjects) + + (d.subjects === 1 ? " person" : " people"); + } + + function update(history) { + if (!Array.isArray(history)) history = []; var clean = []; - for (var i = 0; i < series.length; i++) { - var v = Number(series[i]); - clean.push(isFinite(v) && v > 0 ? v : 0); + for (var i = 0; i < history.length; i++) { + var row = history[i] || {}; + var v = Number(row.requests); + clean.push({ + date: String(row.date || ""), + requests: isFinite(v) && v > 0 ? v : 0, + input_tokens: Number(row.input_tokens) || 0, + output_tokens: Number(row.output_tokens) || 0, + subjects: Number(row.subjects) || 0 + }); } var prevShown = shown.slice(); - target = clean; + days = clean; + if (clean.length) setText("chart-x0", clean[0].date); if (raf) cancelAnimationFrame(raf); + var heights = clean.map(function (d) { return d.requests; }); if (reducedMotion.matches || prevShown.length === 0) { - shown = clean.slice(); + shown = heights; draw(); return; } - // align previous frame to the new series length (both end at "now") + // Both series end at today, so align them from the right: yesterday + // stays yesterday when a new day appears on the end. tweenFrom = []; var shift = clean.length - prevShown.length; for (var j = 0; j < clean.length; j++) { @@ -245,14 +258,12 @@ } canvas.addEventListener("pointermove", function (ev) { - var n = shown.length; - if (n < 2) return; + var n = days.length; + if (n === 0) return; var rect = canvas.getBoundingClientRect(); var frac = (ev.clientX - rect.left) / rect.width; - hoverIdx = Math.max(0, Math.min(n - 1, Math.round(frac * (n - 1)))); - var age = n - 1 - hoverIdx; - setText("spark-readout", Math.round(shown[hoverIdx]) + " tok/s · " + - (age === 0 ? "now" : age + "s ago")); + hoverIdx = Math.max(0, Math.min(n - 1, Math.floor(frac * n))); + setText("spark-readout", readout(hoverIdx)); draw(); }); canvas.addEventListener("pointerleave", function () { @@ -372,6 +383,55 @@ } setInterval(tickCountdown, 1000); + /* ---------- upstream health ---------- + + Two rows, because a visitor whose request just failed has exactly one + question and it is not "what is your p99": is this you or DeepSeek? + Our own row is derived from the gateway's own state word; the DeepSeek + row is what our last calls to api.deepseek.com actually did. */ + + var UP_STATES = { + ok: ["reachable", "ok"], + degraded: ["some calls failing", "warn"], + down: ["not answering us", "crit"], + unknown: ["not called yet", "idle"] + }; + + function setDot(id, cls) { + var el = $(id); + if (el) el.className = "health-dot " + cls; + } + + function renderUpstream(d) { + // Ours: anything that still serves requests is working, and the two + // exhausted states are our limit rather than a fault. + var st = String(d.state || ""); + var usText = "serving requests", usCls = "ok"; + if (st === "degraded") { usText = "refusing — cannot record spend"; usCls = "crit"; } + else if (st === "day_exhausted") { usText = "today's budget spent"; usCls = "warn"; } + else if (st === "credit_exhausted") { usText = "credit pool empty"; usCls = "warn"; } + else if (st === "busy") { usText = "busy — requests queue"; usCls = "warn"; } + else if (st !== "operational") { usText = String(d.state || "unknown"); usCls = "idle"; } + setText("up-us", usText); + setDot("up-us-dot", usCls); + + var u = d.upstream || {}; + var m = UP_STATES[u.state] || UP_STATES.unknown; + setText("up-ds", m[0]); + setDot("up-ds-dot", m[1]); + + var bits = []; + if (u.latency_ms > 0) bits.push("last good call " + fmtInt(u.latency_ms) + " ms"); + if (u.last_ok_ago_sec >= 0) bits.push(fmtDur(u.last_ok_ago_sec) + " ago"); + if (u.fault_streak > 0) { + bits.push(fmtInt(u.fault_streak) + " failing in a row" + + (u.last_fault ? " (" + u.last_fault + ")" : "")); + } else if (u.faults > 0) { + bits.push(fmtInt(u.faults) + " of " + fmtInt(u.calls) + " calls failed since boot"); + } + setText("up-note", bits.length ? bits.join(" · ") : "nothing forwarded since this process started"); + } + /* ---------- render ---------- */ function render(d) { @@ -387,19 +447,29 @@ setText("t-flight", fmtInt(live.in_flight)); var usage = d.usage || {}; + var today = usage.today || {}; + var life = usage.lifetime || {}; setText("t-today", fmtInt(usage.subjects_today)); + setText("t-req-today", fmtInt(today.requests)); + setText("t-req-life", fmtCompact(life.requests)); var sys = d.system || {}; setText("t-uptime", fmtDur(sys.uptime_sec)); - chart.update(live.series); + var history = Array.isArray(d.history) ? d.history : []; + var sum30 = 0; + history.forEach(function (row) { sum30 += Number(row && row.requests) || 0; }); + setText("t-req-30d", fmtCompact(sum30)); + chart.update(history); + setText("chart-now", "right now: " + fmtRate(live.tokens_per_sec) + + " tokens/sec · " + fmtInt(live.in_flight) + " in flight"); + + renderUpstream(d); var credit = d.credit || {}; setGauge("g-day", credit.day_remaining_pct); setGauge("g-pool", credit.pool_remaining_pct); - var today = usage.today || {}; - var life = usage.lifetime || {}; setText("to-req", fmtCompact(today.requests)); setText("lt-req", fmtCompact(life.requests)); setText("to-in", fmtCompact(today.input_tokens)); @@ -430,6 +500,7 @@ setText("lim-req", fmtInt(lim.requests)); setText("lim-in", fmtCompact(lim.input_tokens)); setText("lim-out", fmtCompact(lim.output_tokens)); + setText("lim-search", fmtInt(lim.searches)); var rAt = Date.parse(d.resets_at); var sNow = Date.parse(d.now); diff --git a/gateway/internal/server/web/index.html b/gateway/internal/server/web/index.html index 51a65ea..e5c51bf 100644 --- a/gateway/internal/server/web/index.html +++ b/gateway/internal/server/web/index.html @@ -74,21 +74,21 @@

Live status

-
—tokens/sec
-
—requests/min
-
—live users (5m)
-
—in flight
-
—users today
+
—requests today
+
—people today
+
—requests · 30d
+
—requests · all time
—uptime
- output tokens/sec — last 5 minutes + requests per day — last 30 days
- - + + +

—

@@ -126,14 +126,50 @@

Totals

+
+

Is it us or DeepSeek?

+ +

—

+

Measured from our own calls to + api.deepseek.com, not from a status page — a page cannot see + a route that is broken only from here. DeepSeek's own + incident page is the place to + check for anything wider.

+

Endpoints

+
+ +

Countries

+
+

Right now

+ +

A pool this size is idle most of the day, so these + are usually zero. That is the honest reading, not a broken widget — the + daily chart above is where the traffic shows.

+
@@ -186,7 +222,13 @@

Per-user daily limits

  • 30 requests
  • 60k input tokens
  • 20k output tokens
    • +
  • 3 web searches
    • +

    Server-side web_search works here: + deepseek respond "…" --web-search, or the web_search + tool on /responses from any OpenAI client. It has its own small + ration because one search reads whole pages into the prompt and costs about + what ten ordinary turns cost — see economics.

    diff --git a/gateway/internal/server/web/pages/economics.html b/gateway/internal/server/web/pages/economics.html index 2048ebb..d2b583c 100644 --- a/gateway/internal/server/web/pages/economics.html +++ b/gateway/internal/server/web/pages/economics.html @@ -57,8 +57,51 @@

    The same task

    Not twenty percent cheaper. Thirty to a few hundred times cheaper, depending on the model and how much of the prompt caches.

    +

    That table is still arithmetic on a rate card, which is the weakest kind of cost claim — it assumes every model spends the same tokens on the same work, and they do not. The section below is the stronger evidence: what independent benchmarks actually paid.

    + +

    What the benchmarks actually paid

    +

    Two leaderboards publish a dollar figure next to the score, measured from their own runs rather than multiplied out of a price list. Artificial Analysis reports cost per task — total spend to run its Intelligence Index divided by the number of tasks — alongside the index score itself:

    + + + + + + + + + + + + +
    modelintelligence indexcost per task
    deepseek-v4-flash (max)52$0.03
    deepseek-v4-pro (max)45$0.05
    GPT-5.6 Luna (max)52$0.05
    Gemini 3.6 Flash52$0.56
    GPT-5.6 Sol (max)61$1.23
    Claude Sonnet 5 (max)55$1.72
    Claude Opus 5 (max)63$2.34
    Claude Fable 562$3.14
    +

    The same source publishes what the whole index cost to run per model, which is the least ambiguous number on this page: $72.03 for deepseek-v4-flash against $3,836.05 for Claude Opus 5 and $5,455.22 for Claude Fable 5 — 53× and 76× on identical work.

    + +

    On coding specifically, SWE-rebench runs one fixed harness over 111 real GitHub issues and publishes cost per problem next to the resolve rate:

    + + + + + + + + + + +
    modelresolvedcost per problem
    Claude Fable 5 (high)64.5%$4.40
    Claude Opus 5 (high)63.4%$3.47
    GPT-5.6 Sol (medium)62.3%$0.85
    Claude Sonnet 5 (high)56.8%$1.43
    GPT-5.6 Luna (medium)43.6%$0.11
    deepseek-v4-pro (high)40.2%$0.15
    +

    And one independent developer ran the same build prompt three times back to back in early August 2026 and published the receipts: Claude Opus 5 $0.3183, Kimi K3 $0.2485, deepseek-v4-flash $0.0049. All three produced working output; DeepSeek failed silently on image handling where Claude improvised around bugs that stopped the others.

    + +

    Where the cheap-token argument breaks

    +

    Two numbers in those tables cut against the story this page is telling, and leaving them out would make it propaganda. On Artificial Analysis, GPT-5.6 Luna scores the same 52 as flash for $0.05 against flash's $0.03 — a 1.7× gap, not a hundredfold one. On SWE-rebench, Luna beats deepseek-v4-pro on both axes at once: more issues resolved, less money per issue. So the honest shape of the claim is narrower than "DeepSeek is the cheapest way to get work done":

    + +

    Every figure above was read on 2026-08-07 and they drift for two reasons at once: vendors reprice (DeepSeek's own cost-to-run fell from $1,071 to $176 for v4-pro after a price cut), and benchmarks reweight their indices. Treat the ratios as this month's, not as constants.

    +

    What this does not claim

    -

    Per-token is not per-task. A stronger model that solves a hard problem in one attempt can beat a cheaper model that needs three, and on the hardest work the frontier models earn their price. Tokenizers differ too — Anthropic notes that its current tokenizer produces roughly 30% more tokens for the same text than its previous one, so identical work is not identical token counts across vendors. And DeepSeek has announced peak-hour pricing at 2× the listed rates, effective date not yet set. The honest claim is narrower and still remarkable: for the broad middle of real work — summarize, translate, refactor, answer, glue — the going rate differs by two orders of magnitude depending on whose API you call.

    +

    Three smaller caveats on top of the big one. Tokenizers differ, so identical text is not identical token counts across vendors — Anthropic notes its current tokenizer emits roughly 30% more tokens for the same input than its previous one. DeepSeek has announced peak-hour pricing at 2× the listed rates, with no effective date yet, which would halve the gap during working hours in Beijing. And a harness costs money too: one comparison held flash constant across four agent frameworks and watched cost per successful task swing 2.7× — $0.073 on the cheapest, $0.195 on Claude Code — so the tool you drive the model with can matter as much as the model.

    +

    What survives all of that is still worth saying plainly: for the broad middle of real work — summarize, translate, refactor, answer, glue — the going rate differs by one to two orders of magnitude depending on whose API you call, and the cheapest credible option is no longer a toy.

    Why it matters

    Chat is measured in thousands of tokens; agents are measured in millions. The moment a model works unattended — reading files, retrying, checking its own output — token consumption stops tracking human attention and starts tracking machine patience. An overnight agent run that emits ten million output tokens costs $2.80 at flash prices and $500 at Fable prices. One of those is "leave it running"; the other is a line item that gets a meeting. At frontier prices, autonomy is a luxury good. At flash prices, it is a background process.

    diff --git a/gateway/internal/server/web/style.css b/gateway/internal/server/web/style.css index b0e71a0..c8cd708 100644 --- a/gateway/internal/server/web/style.css +++ b/gateway/internal/server/web/style.css @@ -195,7 +195,7 @@ h3 { font-size: 13px; margin: 0 0 10px; text-transform: uppercase; letter-spacin .tiles { display: grid; - grid-template-columns: repeat(6, 1fr); + grid-template-columns: repeat(5, 1fr); gap: 10px; margin-bottom: 14px; } @@ -234,7 +234,7 @@ h3 { font-size: 13px; margin: 0 0 10px; text-transform: uppercase; letter-spacin #spark { display: block; width: 100%; - height: 150px; + height: 168px; color: var(--cyan); /* line + fill colour, read by app.js */ border-color: var(--muted); /* axis/grid colour, read by app.js */ touch-action: pan-y; @@ -247,6 +247,47 @@ h3 { font-size: 13px; margin: 0 0 10px; text-transform: uppercase; letter-spacin margin-top: 4px; } +/* the live figures, demoted under the chart: true but usually zero */ +.chart-now { + margin: 6px 0 0; + font-size: 11.5px; + color: var(--muted); + font-variant-numeric: tabular-nums; +} + +/* upstream health rows */ +.health { list-style: none; margin: 0 0 10px; padding: 0; } +.health li { + display: grid; + grid-template-columns: 10px 1fr auto; + align-items: center; + gap: 10px; + padding: 7px 0; + border-bottom: 1px solid var(--line); +} +.health li:last-child { border-bottom: 0; } +.health-who { color: var(--text); font-size: 13px; } +.health-state { + color: var(--muted); + font-size: 12.5px; + text-align: right; + font-variant-numeric: tabular-nums; +} +.health-dot { + width: 8px; + height: 8px; + border-radius: 50%; + background: var(--muted); + box-shadow: 0 0 0 3px color-mix(in srgb, var(--muted) 18%, transparent); +} +/* Colour is a second signal here, never the only one: the words next to + each dot say the same thing, so the panel survives being read in + greyscale or by someone who cannot separate red from green. */ +.health-dot.ok { background: var(--cyan); box-shadow: 0 0 0 3px color-mix(in srgb, var(--cyan) 20%, transparent); } +.health-dot.warn { background: var(--yellow); box-shadow: 0 0 0 3px color-mix(in srgb, var(--yellow) 20%, transparent); } +.health-dot.crit { background: var(--pink); box-shadow: 0 0 0 3px color-mix(in srgb, var(--pink) 20%, transparent); } +.health-dot.idle { opacity: 0.55; } + /* two-column card rows */ .cols { display: grid; diff --git a/gateway/internal/stats/stats.go b/gateway/internal/stats/stats.go index 7454bf2..f9977f2 100644 --- a/gateway/internal/stats/stats.go +++ b/gateway/internal/stats/stats.go @@ -47,6 +47,27 @@ type bucket struct { output int64 } +// Upstream is what the gateway has observed of DeepSeek itself. +type Upstream struct { + // State is one word: ok, degraded (some calls failing) or down + // (nothing has succeeded since the last failure). + State string `json:"state"` + // LastOKAgoSec and LastFaultAgoSec are ages rather than timestamps so + // a reader does not have to trust our clock against theirs. Negative + // means it has not happened in this process's lifetime. + LastOKAgoSec int64 `json:"last_ok_ago_sec"` + LastFaultAgoSec int64 `json:"last_fault_ago_sec"` + // LastFault is the shape of the last failure — a status code or a + // transport error class. Never a URL, a key or a prompt. + LastFault string `json:"last_fault,omitempty"` + Calls int64 `json:"calls"` + Faults int64 `json:"faults"` + // FaultStreak is consecutive failures right now. It is the number that + // distinguishes "one bad request" from "DeepSeek is having an outage". + FaultStreak int `json:"fault_streak"` + LatencyMS int64 `json:"latency_ms"` +} + // Collector accumulates the live view. type Collector struct { mu sync.Mutex @@ -66,6 +87,19 @@ type Collector struct { inFlight int64 started time.Time + // Upstream health: what DeepSeek did the last time we spoke to it. + // This is the honest answer to "is it you or them", and it is a + // first-party observation rather than a status page — a vendor's + // status page cannot see a route that is broken only from here, and + // DeepSeek's is not reachable from outside China anyway. + upLastOK time.Time + upLastFault time.Time + upFaultWhat string + upCalls int64 + upFaults int64 + upStreak int + upLatencyMS int64 + now func() time.Time } @@ -191,10 +225,11 @@ type System struct { // Snapshot is everything the dashboard needs from this package. type Snapshot struct { - Live Live `json:"live"` - Endpoints []Count `json:"endpoints"` - Countries []Count `json:"countries"` - System System `json:"system"` + Live Live `json:"live"` + Endpoints []Count `json:"endpoints"` + Countries []Count `json:"countries"` + System System `json:"system"` + Upstream Upstream `json:"upstream"` } // Snapshot reads the live view. Cheap enough to call per request, but the @@ -234,6 +269,7 @@ func (c *Collector) Snapshot() Snapshot { }, Endpoints: topLocked(c.endpoints, 10), Countries: topLocked(c.countries, 12), + Upstream: c.upstreamLocked(), System: System{ UptimeSec: int64(c.now().Sub(c.started).Seconds()), Load1: loadAvg1(), @@ -297,3 +333,63 @@ func loadAvg1() float64 { } return 0 } + +// Upstream records one round trip to DeepSeek. +// +// fault is the caller's judgement, not a status code test, because the +// gateway already distinguishes "DeepSeek is broken" (429, 5xx, transport +// failure) from "the caller's own request came back 4xx" — and only the +// first is an upstream fault. Counting a user's malformed JSON as a +// DeepSeek outage would make this panel lie in the most misleading +// direction. +func (c *Collector) Upstream(what string, fault bool, latency time.Duration) { + c.mu.Lock() + defer c.mu.Unlock() + now := c.now() + c.upCalls++ + if fault { + c.upFaults++ + c.upStreak++ + c.upLastFault = now + c.upFaultWhat = what + return + } + c.upStreak = 0 + c.upLastOK = now + if ms := latency.Milliseconds(); ms >= 0 { + c.upLatencyMS = ms + } +} + +// upstreamLocked reduces the counters to the shape the dashboard reads. +func (c *Collector) upstreamLocked() Upstream { + now := c.now() + ago := func(t time.Time) int64 { + if t.IsZero() { + return -1 + } + return int64(now.Sub(t).Seconds()) + } + u := Upstream{ + State: "ok", + LastOKAgoSec: ago(c.upLastOK), + LastFaultAgoSec: ago(c.upLastFault), + LastFault: c.upFaultWhat, + Calls: c.upCalls, + Faults: c.upFaults, + FaultStreak: c.upStreak, + LatencyMS: c.upLatencyMS, + } + switch { + case c.upCalls == 0: + // Nothing has been asked of DeepSeek since this process started, so + // there is nothing to report either way. Saying "ok" would be a + // guess dressed as a measurement. + u.State = "unknown" + case c.upStreak >= 3: + u.State = "down" + case c.upStreak > 0: + u.State = "degraded" + } + return u +} diff --git a/internal/cli/free.go b/internal/cli/free.go index 23f769e..9ed1f2a 100644 --- a/internal/cli/free.go +++ b/internal/cli/free.go @@ -142,8 +142,14 @@ func (o *Options) printFreeOffer(gateway string, info *deepseek.FreeInfo) { row := func(k, v string) { fmt.Fprintf(w, " %-9s %s\n", k, v) } row("gateway", gateway) row("model", info.Model) - row("per day", fmt.Sprintf("%s requests · %s input · %s output tokens", - humanTokens(info.Limits.Requests), humanTokens(info.Limits.InputTokens), humanTokens(info.Limits.OutputTokens))) + perDay := fmt.Sprintf("%s requests · %s input · %s output tokens", + humanTokens(info.Limits.Requests), humanTokens(info.Limits.InputTokens), humanTokens(info.Limits.OutputTokens)) + // A gateway that does not offer web search sends no ration, and + // printing "0 searches" would read as "you have used them all". + if info.Limits.Searches > 0 { + perDay += fmt.Sprintf(" · %d web searches", info.Limits.Searches) + } + row("per day", perDay) if info.MaxTokens > 0 { row("per call", fmt.Sprintf("%s output tokens max", humanTokens(info.MaxTokens))) } @@ -187,10 +193,14 @@ func runFreeStatus(ctx context.Context, o *Options) error { row("gateway", free.BaseURL) row("subject", free.Subject) row("enrolled", free.Enrolled.Local().Format("2006-01-02")) - row("today", fmt.Sprintf("%d/%d requests · %s/%s in · %s/%s out", + today := fmt.Sprintf("%d/%d requests · %s/%s in · %s/%s out", quota.Used.Requests, quota.Limits.Requests, humanTokens(quota.Used.InputTokens), humanTokens(quota.Limits.InputTokens), - humanTokens(quota.Used.OutputTokens), humanTokens(quota.Limits.OutputTokens))) + humanTokens(quota.Used.OutputTokens), humanTokens(quota.Limits.OutputTokens)) + if quota.Limits.Searches > 0 { + today += fmt.Sprintf(" · %d/%d searches", quota.Used.Searches, quota.Limits.Searches) + } + row("today", today) row("spent", money(quota.Used.SpentUSD)+" "+o.dim("(on our credits, not yours)")) if d := time.Until(quota.ResetsAt); d > 0 { row("resets", fmt.Sprintf("in %s %s", roundDuration(d), o.dim("(00:00 UTC)"))) diff --git a/internal/deepseek/free.go b/internal/deepseek/free.go index f8a8497..e939f76 100644 --- a/internal/deepseek/free.go +++ b/internal/deepseek/free.go @@ -116,6 +116,10 @@ type FreeInfo struct { Requests int `json:"requests"` InputTokens int `json:"input_tokens"` OutputTokens int `json:"output_tokens"` + // Searches is the daily ration for server-side web_search. Older + // gateways do not send it, so zero means "not offered" rather than + // "none left". + Searches int `json:"searches"` } `json:"daily_limits"` Endpoints []string `json:"endpoints"` } @@ -128,12 +132,14 @@ type FreeQuota struct { Requests int `json:"requests"` InputTokens int `json:"input_tokens"` OutputTokens int `json:"output_tokens"` + Searches int `json:"searches"` SpentUSD float64 `json:"spent_usd"` } `json:"used"` Limits struct { Requests int `json:"requests"` InputTokens int `json:"input_tokens"` OutputTokens int `json:"output_tokens"` + Searches int `json:"searches"` } `json:"limits"` ResetsAt time.Time `json:"resets_at"` Exhausted bool `json:"service_exhausted"` diff --git a/site/build.py b/site/build.py index 8c97450..3061697 100644 --- a/site/build.py +++ b/site/build.py @@ -1405,6 +1405,16 @@ def jstr(s):

    + +

    + +