SCAI estimates a prospect's dedicated ML inference spend from public signals for AI infrastructure teams.
I spent four months on an ML platform team whose GPU cluster sat at 5% utilisation. The instinct was to add nodes, but a 2-second feature query sat in front of a 1-millisecond inference: the hardware was starving, not saturated. That waste was invisible from the outside, and it has been invisible at every company I have looked at since. SCAI estimates it from public signals alone.
Landing state: analyse one company or upload a CSV.
Signal extraction: each claim retains its evidence quote, confidence, and source URL.
Cost estimate: the full editable assumption set and unverified pricing row remain visible.
Email draft: validated cost figures, a linked signal, and a visible word count.
Negative state: no grounded public signal means no estimate and no email.
These captures come from the recorded fixture in scripts/fixtures/capture.json; python scripts/capture.py reproduces them without an API key or model call.
flowchart LR
A["1 · Signal extraction<br/>Gemini + Google Search grounding"] -->|"structured JSON<br/>source URL on every claim"| B["2 · Cost estimate<br/>pure Python + pricing.json"]
B -->|"validated numbers only"| C["3 · Email draft<br/>Gemini constrained to Stage 2 output"]
Stage 1 retrieves public evidence and validates it into typed JSON. Stage 2 is a deterministic Python function with no model call. Stage 3 receives the validated signals and the completed cost object; its output is rejected if it introduces a dollar figure outside that object.
The LLM is used for retrieval and language, never for arithmetic. Every dollar figure is computed in app/cost.py from a hand-maintained pricing.json catalog whose rows carry source URLs and whose top-level as_of date records when the prices were checked.
Any pricing row that could not be verified is marked TODO_VERIFY. The cost engine rejects it, and the interface greys it out instead of guessing. A hallucinated price would make the entire tool worthless; separating probabilistic retrieval and drafting from deterministic cost math is the design decision SCAI is built around.
Let q be peak queries per second, t throughput per instance, p hourly instance price, h hours per month, u utilisation as a percentage, and a API price per 1,000 inferences.
| Assumption | Default value | Where it comes from | How to override it |
|---|---|---|---|
| Scale band | Signal result; unknown falls back to medium |
Stage 1 structured output; fallback in app/cost.py |
Select small, medium, or large in the cost table, or change inferred.scale_band in POST /api/estimate |
Peak QPS (q) |
small: 5, medium: 50, large: 500, unknown: 50 |
DEFAULT_PEAK_QPS in app/cost.py |
Edit Peak QPS, or set overrides.peak_qps |
| Estimate range | q_low = 0.5q; q_high = 1.5q |
Fixed range in app/cost.py |
Not exposed; change the two multipliers in app/cost.py |
Utilisation (u) |
15% |
DEFAULT_UTILISATION_PCT in app/cost.py |
Edit Assumed utilization, or set overrides.assumed_utilisation_pct |
Hours per month (h) |
730 |
DEFAULT_HOURS_PER_MONTH in app/cost.py |
Edit Hours per month, or set overrides.hours_per_month |
Throughput per instance (t) |
small: 4, medium: 20, large: 100 inferences/s |
Planning bands in the selected pricing.json row; they are not benchmarks |
Edit Throughput / instance, set overrides.throughput_inferences_per_sec, or maintain the catalog |
Dedicated instance (p) |
GCP g2-standard-4 with one NVIDIA L4 in us-central1, $0.706832276/hour |
Verified public list-rate row in pricing.json |
Select another verified row, set overrides.instance_id, or maintain the catalog |
API comparison (a) |
$3.75 / 1,000 inferences |
pricing.json: 1,000 input and 250 output tokens at the catalogued public list rates |
Edit Per-call API / 1k, set overrides.api_price_per_1k_inferences_usd, or maintain the catalog |
| Seconds per hour | 3,600 |
Unit conversion in app/cost.py |
Not configurable |
The engine applies the same equations at q_low, q, and q_high where a range is required:
instances_needed(q) = ceil(q / t)
monthly_cost(q) = instances_needed(q) × p × h
idle_spend(q) = monthly_cost(q) × (1 − u / 100)
monthly_inferences(q) = q × h × 3,600 × (u / 100)
cost_per_1k(q) = monthly_cost(q) / monthly_inferences(q) × 1,000
api_cost_at_full_utilisation(q) = q × h × 3,600 / 1,000 × a
breakeven_utilisation(q) = min(100, monthly_cost(q) / api_cost_at_full_utilisation(q) × 100)
For the default medium band, for example, instances_needed(50) = ceil(50 / 20) = 3; the rest can be reproduced by substituting the table values above. Workload type and model-family guesses do not enter the cost equations.
Python 3.11 or later and Docker are required. Create .env from the tracked template and set GEMINI_API_KEY before starting the container.
git clone https://github.com/daetan999/SCAI.git
cd SCAI
cp .env.example .env
docker build -t scai .
docker run --rm --env-file .env -p 8080:8080 scaiOpen http://localhost:8080.
The included deploy script builds from source and deploys the scai service to Cloud Run:
export GOOGLE_CLOUD_PROJECT="your-project-id"
bash scripts/deploy_cloud_run.shIt defaults to asia-southeast1; set CLOUD_RUN_REGION to override the region. The deploying identity needs permission to build and deploy Cloud Run services and to use the configured Gemini endpoint.
Upload a UTF-8 CSV whose first column contains one company name or public careers-page URL per row. The optional header may be company, company_name, or url; a batch is limited to 50 rows and 1 MB.
company
Google Cloud
NVIDIAThe response columns are:
company,serves_models,idle_spend_low,idle_spend_high,top_signal_url,subject,email_body,no_signal_found
With the app running, submit the tracked example file and save the response:
curl -fsS -F 'file=@sample_companies.csv' http://localhost:8080/api/batch -o scai-batch-results.csv- Estimates are ranges derived from public list rates. They ignore committed-use discounts, negotiated pricing, spot capacity, taxes, storage, networking, support, and engineering labour.
- Throughput figures are coarse planning bands, not benchmarks. Model architecture, precision, batch size, sequence length, hardware, and latency targets can move the result materially.
- Signal extraction depends on a company publishing something indexable. Absence of signal is not absence of workload.
- SCAI does not detect batch versus real-time inference reliably. A wrong workload classification makes the chosen throughput band less useful.
- The default assumes dedicated, continuously provisioned instances. Autoscaling, shared clusters, serverless endpoints, queues, and burst patterns are not modelled.
- Source grounding constrains citations; it does not prove that a source is current, complete, or representative of the deployed system.
- The API comparison uses a fixed token-shape assumption. It is not a quote for a prospect's actual traffic.
SCAI is not affiliated with, endorsed by, or representing any cloud or GPU vendor. Pricing data is from public list pages and is accurate only as of the as_of date in pricing.json.