Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: Cost tests

on:
push:
pull_request:

permissions:
contents: read

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
cache: pip
- name: Install test dependencies
run: python -m pip install -r requirements-dev.txt
- name: Run deterministic cost tests
run: >-
python -m pytest -q tests/test_cost.py
--cov=app.cost --cov-report=term-missing --cov-fail-under=80
188 changes: 98 additions & 90 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,137 +1,145 @@
![SCAI — Signal & Cost engine for AI Infrastructure](docs/banner.svg)

# SCAI

**Public production-ML signals → deterministic inference cost range → evidence-led outreach.**
SCAI estimates a prospect's dedicated ML inference spend from public signals for AI infrastructure teams.

SCAI accepts a company name or careers-page URL, uses Gemini with Google Search grounding to find evidence of production ML inference, calculates a transparent dedicated-infrastructure cost range in pure Python, and drafts a cold email that can only reference validated signals and calculated numbers.
![SCAI fixture-backed demo](docs/demo.gif)

All prices, throughput assumptions, utilization inputs, sources, and dates are visible. If no usable grounded source is returned, SCAI says to skip the account and produces no estimate or email.
[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-3776AB?logo=python&logoColor=white)](Dockerfile)
[![Cost tests](https://github.com/daetan999/SCAI/actions/workflows/tests.yml/badge.svg)](https://github.com/daetan999/SCAI/actions/workflows/tests.yml)

## Model and SDK
## The problem

- Default model: `gemini-3.5-flash`
- Override: `GEMINI_MODEL`
- Python package: `google-genai`
- Imports: `from google import genai` and `from google.genai import types`
I spent four months on an ML platform team whose GPU cluster sat at 5% utilisation. The instinct was to add nodes, but a 2-second feature query sat in front of a 1-millisecond inference: the hardware was starving, not saturated. That waste was invisible from the outside, and it has been invisible at every company I have looked at since. SCAI estimates it from public signals alone.

Gemini 3.5 Flash was selected because current Google documentation lists support for Google Search grounding and structured output with built-in tools. The model remains configurable because temporary workshop projects can expose different model catalogs.
## Demo

## Architecture
![Landing state with company input and batch upload](docs/01-empty.png)

```text
Stage 1: Gemini + Google Search grounding
company → grounded public signals → validated SignalResult
*Landing state: analyse one company or upload a CSV.*

Stage 2: Pure Python
SignalResult + pricing.json + visible overrides → deterministic CostEstimate
![Grounded signal cards with evidence quotes and source links](docs/02-signals.png)

Stage 3: Gemini without grounding
validated signals + CostEstimate → constrained EmailDraft
```
*Signal extraction: each claim retains its evidence quote, confidence, and source URL.*

No model output can create an hourly price or calculate a cost. Stage 2 only accepts the hand-edited `pricing.json` catalog and typed user overrides.
![Deterministic cost estimate with the assumptions table expanded](docs/03-cost.png)

## Run locally
*Cost estimate: the full editable assumption set and unverified pricing row remain visible.*

Python 3.11 or later is required.
![Constrained draft email with word counter and referenced signal](docs/04-email.png)

```bash
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements-dev.txt
export GEMINI_API_KEY="YOUR_KEY"
export GEMINI_MODEL="gemini-3.5-flash"
python -m uvicorn app.main:app --reload --port 8080
```
*Email draft: validated cost figures, a linked signal, and a visible word count.*

Open [http://localhost:8080](http://localhost:8080).
![No public signal refusal state](docs/05-no-signal.png)

Run the cost tests:
*Negative state: no grounded public signal means no estimate and no email.*

```bash
python -m pytest -q
These captures come from the recorded fixture in `scripts/fixtures/capture.json`; `python scripts/capture.py` reproduces them without an API key or model call.

## How it works

```mermaid
flowchart LR
A["1 · Signal extraction<br/>Gemini + Google Search grounding"] -->|"structured JSON<br/>source URL on every claim"| B["2 · Cost estimate<br/>pure Python + pricing.json"]
B -->|"validated numbers only"| C["3 · Email draft<br/>Gemini constrained to Stage 2 output"]
```

## Google Cloud / Vertex AI
Stage 1 retrieves public evidence and validates it into typed JSON. Stage 2 is a deterministic Python function with no model call. Stage 3 receives the validated signals and the completed cost object; its output is rejected if it introduces a dollar figure outside that object.

The same package works in Cloud Shell and Cloud Run with Application Default Credentials:
## Why the model never produces a number

```bash
export GOOGLE_CLOUD_PROJECT="$(gcloud config get-value project)"
export GOOGLE_CLOUD_LOCATION="global"
export GOOGLE_GENAI_USE_VERTEXAI="True"
export GOOGLE_GENAI_USE_ENTERPRISE="True"
export GEMINI_MODEL="gemini-3.5-flash"
export APP_MODE="live"
The LLM is used for retrieval and language, never for arithmetic. Every dollar figure is computed in `app/cost.py` from a hand-maintained `pricing.json` catalog whose rows carry source URLs and whose top-level `as_of` date records when the prices were checked.

Any pricing row that could not be verified is marked `TODO_VERIFY`. The cost engine rejects it, and the interface greys it out instead of guessing. A hallucinated price would make the entire tool worthless; separating probabilistic retrieval and drafting from deterministic cost math is the design decision SCAI is built around.

## Cost model

Let `q` be peak queries per second, `t` throughput per instance, `p` hourly instance price, `h` hours per month, `u` utilisation as a percentage, and `a` API price per 1,000 inferences.

| Assumption | Default value | Where it comes from | How to override it |
|---|---:|---|---|
| Scale band | Signal result; `unknown` falls back to `medium` | Stage 1 structured output; fallback in `app/cost.py` | Select `small`, `medium`, or `large` in the cost table, or change `inferred.scale_band` in `POST /api/estimate` |
| Peak QPS (`q`) | `small: 5`, `medium: 50`, `large: 500`, `unknown: 50` | `DEFAULT_PEAK_QPS` in `app/cost.py` | Edit **Peak QPS**, or set `overrides.peak_qps` |
| Estimate range | `q_low = 0.5q`; `q_high = 1.5q` | Fixed range in `app/cost.py` | Not exposed; change the two multipliers in `app/cost.py` |
| Utilisation (`u`) | `15%` | `DEFAULT_UTILISATION_PCT` in `app/cost.py` | Edit **Assumed utilization**, or set `overrides.assumed_utilisation_pct` |
| Hours per month (`h`) | `730` | `DEFAULT_HOURS_PER_MONTH` in `app/cost.py` | Edit **Hours per month**, or set `overrides.hours_per_month` |
| Throughput per instance (`t`) | `small: 4`, `medium: 20`, `large: 100` inferences/s | Planning bands in the selected `pricing.json` row; they are not benchmarks | Edit **Throughput / instance**, set `overrides.throughput_inferences_per_sec`, or maintain the catalog |
| Dedicated instance (`p`) | GCP `g2-standard-4` with one NVIDIA L4 in `us-central1`, `$0.706832276/hour` | Verified public list-rate row in `pricing.json` | Select another verified row, set `overrides.instance_id`, or maintain the catalog |
| API comparison (`a`) | `$3.75 / 1,000` inferences | `pricing.json`: 1,000 input and 250 output tokens at the catalogued public list rates | Edit **Per-call API / 1k**, set `overrides.api_price_per_1k_inferences_usd`, or maintain the catalog |
| Seconds per hour | `3,600` | Unit conversion in `app/cost.py` | Not configurable |

The engine applies the same equations at `q_low`, `q`, and `q_high` where a range is required:

```text
instances_needed(q) = ceil(q / t)
monthly_cost(q) = instances_needed(q) × p × h
idle_spend(q) = monthly_cost(q) × (1 − u / 100)

monthly_inferences(q) = q × h × 3,600 × (u / 100)
cost_per_1k(q) = monthly_cost(q) / monthly_inferences(q) × 1,000

api_cost_at_full_utilisation(q) = q × h × 3,600 / 1,000 × a
breakeven_utilisation(q) = min(100, monthly_cost(q) / api_cost_at_full_utilisation(q) × 100)
```

Validate the package and one grounded request:
For the default medium band, for example, `instances_needed(50) = ceil(50 / 20) = 3`; the rest can be reproduced by substituting the table values above. Workload type and model-family guesses do not enter the cost equations.

## Quickstart

Python 3.11 or later and Docker are required. Create `.env` from the tracked template and set `GEMINI_API_KEY` before starting the container.

```bash
python scripts/validate_environment.py
python scripts/validate_environment.py --live
git clone https://github.com/daetan999/SCAI.git
cd SCAI
cp .env.example .env
docker build -t scai .
docker run --rm --env-file .env -p 8080:8080 scai
```

Deploy from source using the included non-root Dockerfile:
Open [http://localhost:8080](http://localhost:8080).

The included deploy script builds from source and deploys the `scai` service to Cloud Run:

```bash
export GOOGLE_CLOUD_PROJECT="your-project-id"
bash scripts/deploy_cloud_run.sh
```

See [runbook.md](runbook.md) for the full event-day upload, validation, Web Preview, Cloud Run, batch, fallback, and screenshot procedure.
It defaults to `asia-southeast1`; set `CLOUD_RUN_REGION` to override the region. The deploying identity needs permission to build and deploy Cloud Run services and to use the configured Gemini endpoint.

## API
## Batch mode

- `POST /api/analyse` — `{"company": "...", "overrides": {...}}`
- `POST /api/estimate` — reruns Stage 2 only; no search and no model call
- `POST /api/batch` — CSV upload, one company or URL per row, maximum 50
- `GET /api/pricing` — current `pricing.json`
- `GET /health` — `{"ok": true}`
Upload a UTF-8 CSV whose first column contains one company name or public careers-page URL per row. The optional header may be `company`, `company_name`, or `url`; a batch is limited to 50 rows and 1 MB.

Batch output columns:

```text
company,serves_models,idle_spend_low,idle_spend_high,top_signal_url,subject,email_body,no_signal_found
```csv
company
Google Cloud
NVIDIA
```

## Assumptions

Every assumption is visible and editable in the cost panel. Editing the table calls only `/api/estimate`; it does not re-run Google Search.

| Assumption | Default | Basis |
|---|---:|---|
| Utilization | 15% | Hackathon product specification. This is deliberately conservative and user-overridable. |
| Peak QPS — small | 5 | Hackathon product specification. |
| Peak QPS — medium | 50 | Hackathon product specification. |
| Peak QPS — large | 500 | Hackathon product specification. |
| Hours per month | 730 | Hackathon product specification; average month approximation. |
| Low/high range | Peak QPS −50% / +50% | Hackathon product specification. |
| Enabled instance | GCP `g2-standard-4` with one NVIDIA L4 in `us-central1` | Public on-demand list rate from the official [accelerator-optimized VM pricing page](https://cloud.google.com/products/compute/pricing/accelerator-optimized), verified 2026-08-04. |
| Throughput proxy | 4 / 20 / 100 inferences per second for small / medium / large workload bands | Editable planning placeholders for the demo, not provider benchmarks. Replace with a benchmark for the customer's model, precision, batch size, sequence length, and latency target. |
| Per-call comparison | $3.75 per 1,000 inferences | Derived from official [Gemini 3.5 Flash global standard pricing](https://cloud.google.com/gemini-enterprise-agent-platform/generative-ai/pricing): 1,000 input tokens and 250 output tokens per inference at $1.50/M input and $9.00/M output tokens. Token counts are visible modeling assumptions, not a claim about the prospect. |

### Cost formulas
The response columns are:

```text
instances_needed = ceil(peak_qps / throughput_per_instance)
monthly_cost = instances_needed × hourly_usd × hours_per_month
idle_spend = monthly_cost × (1 − utilization)
cost_per_1k = monthly_cost / estimated_monthly_inferences × 1,000
breakeven_utilization = dedicated_monthly_cost / API_cost_at_full_utilization
company,serves_models,idle_spend_low,idle_spend_high,top_signal_url,subject,email_body,no_signal_found
```

The estimate excludes storage, networking, data transfer, support, taxes, committed-use discounts, reserved capacity, autoscaling behavior, engineering labor, and model-specific optimization. It is a discovery hypothesis, not a quote.

## Pricing integrity
With the app running, submit the tracked example file and save the response:

`pricing.json` is intentionally hand-editable. A row with `hourly_usd: null` or `TODO_VERIFY: true` is disabled in the interface and rejected by the cost engine. A visible TODO is safer than a fabricated rate.
```bash
curl -fsS -F 'file=@sample_companies.csv' http://localhost:8080/api/batch -o scai-batch-results.csv
```

Pricing verified as of `2026-08-04`. Recheck official pages before external use.
## Limitations

## Scope
- Estimates are ranges derived from public list rates. They ignore committed-use discounts, negotiated pricing, spot capacity, taxes, storage, networking, support, and engineering labour.
- Throughput figures are coarse planning bands, not benchmarks. Model architecture, precision, batch size, sequence length, hardware, and latency targets can move the result materially.
- Signal extraction depends on a company publishing something indexable. Absence of signal is not absence of workload.
- SCAI does not detect batch versus real-time inference reliably. A wrong workload classification makes the chosen throughput band less useful.
- The default assumes dedicated, continuously provisioned instances. Autoscaling, shared clusters, serverless endpoints, queues, and burst patterns are not modelled.
- Source grounding constrains citations; it does not prove that a source is current, complete, or representative of the deployed system.
- The API comparison uses a fixed token-shape assumption. It is not a quote for a prospect's actual traffic.

- Drafts email only; it never sends.
- Google Search grounding only; no direct website scraping.
- No database, authentication, user accounts, CRM, or stored request state.
- Maximum batch size: 50 rows with a three-request in-process guard.
## Disclaimer

This is a hackathon prototype. Review every source, quote, assumption, and drafted claim before using it.
SCAI is not affiliated with, endorsed by, or representing any cloud or GPU vendor. Pricing data is from public list pages and is accurate only as of the `as_of` date in `pricing.json`.
Binary file added docs/01-empty.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/02-signals.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/03-cost.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/04-email.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/05-no-signal.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
20 changes: 20 additions & 0 deletions docs/banner.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/demo.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions requirements-dev.txt
Original file line number Diff line number Diff line change
@@ -1,2 +1,5 @@
-r requirements.txt
pytest==9.1.1
pytest-cov==6.3.0
playwright==1.61.0
Pillow==12.3.0
Loading
Loading