diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml new file mode 100644 index 0000000..2667d61 --- /dev/null +++ b/.github/workflows/tests.yml @@ -0,0 +1,24 @@ +name: Cost tests + +on: + push: + pull_request: + +permissions: + contents: read + +jobs: + test: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + cache: pip + - name: Install test dependencies + run: python -m pip install -r requirements-dev.txt + - name: Run deterministic cost tests + run: >- + python -m pytest -q tests/test_cost.py + --cov=app.cost --cov-report=term-missing --cov-fail-under=80 diff --git a/README.md b/README.md index b66ceec..736c9ba 100644 --- a/README.md +++ b/README.md @@ -1,137 +1,145 @@ +![SCAI — Signal & Cost engine for AI Infrastructure](docs/banner.svg) + # SCAI -**Public production-ML signals → deterministic inference cost range → evidence-led outreach.** +SCAI estimates a prospect's dedicated ML inference spend from public signals for AI infrastructure teams. -SCAI accepts a company name or careers-page URL, uses Gemini with Google Search grounding to find evidence of production ML inference, calculates a transparent dedicated-infrastructure cost range in pure Python, and drafts a cold email that can only reference validated signals and calculated numbers. +![SCAI fixture-backed demo](docs/demo.gif) -All prices, throughput assumptions, utilization inputs, sources, and dates are visible. If no usable grounded source is returned, SCAI says to skip the account and produces no estimate or email. +[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-3776AB?logo=python&logoColor=white)](Dockerfile) +[![Cost tests](https://github.com/daetan999/SCAI/actions/workflows/tests.yml/badge.svg)](https://github.com/daetan999/SCAI/actions/workflows/tests.yml) -## Model and SDK +## The problem -- Default model: `gemini-3.5-flash` -- Override: `GEMINI_MODEL` -- Python package: `google-genai` -- Imports: `from google import genai` and `from google.genai import types` +I spent four months on an ML platform team whose GPU cluster sat at 5% utilisation. The instinct was to add nodes, but a 2-second feature query sat in front of a 1-millisecond inference: the hardware was starving, not saturated. That waste was invisible from the outside, and it has been invisible at every company I have looked at since. SCAI estimates it from public signals alone. -Gemini 3.5 Flash was selected because current Google documentation lists support for Google Search grounding and structured output with built-in tools. The model remains configurable because temporary workshop projects can expose different model catalogs. +## Demo -## Architecture +![Landing state with company input and batch upload](docs/01-empty.png) -```text -Stage 1: Gemini + Google Search grounding - company → grounded public signals → validated SignalResult +*Landing state: analyse one company or upload a CSV.* -Stage 2: Pure Python - SignalResult + pricing.json + visible overrides → deterministic CostEstimate +![Grounded signal cards with evidence quotes and source links](docs/02-signals.png) -Stage 3: Gemini without grounding - validated signals + CostEstimate → constrained EmailDraft -``` +*Signal extraction: each claim retains its evidence quote, confidence, and source URL.* -No model output can create an hourly price or calculate a cost. Stage 2 only accepts the hand-edited `pricing.json` catalog and typed user overrides. +![Deterministic cost estimate with the assumptions table expanded](docs/03-cost.png) -## Run locally +*Cost estimate: the full editable assumption set and unverified pricing row remain visible.* -Python 3.11 or later is required. +![Constrained draft email with word counter and referenced signal](docs/04-email.png) -```bash -python3 -m venv .venv -source .venv/bin/activate -python -m pip install -r requirements-dev.txt -export GEMINI_API_KEY="YOUR_KEY" -export GEMINI_MODEL="gemini-3.5-flash" -python -m uvicorn app.main:app --reload --port 8080 -``` +*Email draft: validated cost figures, a linked signal, and a visible word count.* -Open [http://localhost:8080](http://localhost:8080). +![No public signal refusal state](docs/05-no-signal.png) -Run the cost tests: +*Negative state: no grounded public signal means no estimate and no email.* -```bash -python -m pytest -q +These captures come from the recorded fixture in `scripts/fixtures/capture.json`; `python scripts/capture.py` reproduces them without an API key or model call. + +## How it works + +```mermaid +flowchart LR + A["1 · Signal extraction
Gemini + Google Search grounding"] -->|"structured JSON
source URL on every claim"| B["2 · Cost estimate
pure Python + pricing.json"] + B -->|"validated numbers only"| C["3 · Email draft
Gemini constrained to Stage 2 output"] ``` -## Google Cloud / Vertex AI +Stage 1 retrieves public evidence and validates it into typed JSON. Stage 2 is a deterministic Python function with no model call. Stage 3 receives the validated signals and the completed cost object; its output is rejected if it introduces a dollar figure outside that object. -The same package works in Cloud Shell and Cloud Run with Application Default Credentials: +## Why the model never produces a number -```bash -export GOOGLE_CLOUD_PROJECT="$(gcloud config get-value project)" -export GOOGLE_CLOUD_LOCATION="global" -export GOOGLE_GENAI_USE_VERTEXAI="True" -export GOOGLE_GENAI_USE_ENTERPRISE="True" -export GEMINI_MODEL="gemini-3.5-flash" -export APP_MODE="live" +The LLM is used for retrieval and language, never for arithmetic. Every dollar figure is computed in `app/cost.py` from a hand-maintained `pricing.json` catalog whose rows carry source URLs and whose top-level `as_of` date records when the prices were checked. + +Any pricing row that could not be verified is marked `TODO_VERIFY`. The cost engine rejects it, and the interface greys it out instead of guessing. A hallucinated price would make the entire tool worthless; separating probabilistic retrieval and drafting from deterministic cost math is the design decision SCAI is built around. + +## Cost model + +Let `q` be peak queries per second, `t` throughput per instance, `p` hourly instance price, `h` hours per month, `u` utilisation as a percentage, and `a` API price per 1,000 inferences. + +| Assumption | Default value | Where it comes from | How to override it | +|---|---:|---|---| +| Scale band | Signal result; `unknown` falls back to `medium` | Stage 1 structured output; fallback in `app/cost.py` | Select `small`, `medium`, or `large` in the cost table, or change `inferred.scale_band` in `POST /api/estimate` | +| Peak QPS (`q`) | `small: 5`, `medium: 50`, `large: 500`, `unknown: 50` | `DEFAULT_PEAK_QPS` in `app/cost.py` | Edit **Peak QPS**, or set `overrides.peak_qps` | +| Estimate range | `q_low = 0.5q`; `q_high = 1.5q` | Fixed range in `app/cost.py` | Not exposed; change the two multipliers in `app/cost.py` | +| Utilisation (`u`) | `15%` | `DEFAULT_UTILISATION_PCT` in `app/cost.py` | Edit **Assumed utilization**, or set `overrides.assumed_utilisation_pct` | +| Hours per month (`h`) | `730` | `DEFAULT_HOURS_PER_MONTH` in `app/cost.py` | Edit **Hours per month**, or set `overrides.hours_per_month` | +| Throughput per instance (`t`) | `small: 4`, `medium: 20`, `large: 100` inferences/s | Planning bands in the selected `pricing.json` row; they are not benchmarks | Edit **Throughput / instance**, set `overrides.throughput_inferences_per_sec`, or maintain the catalog | +| Dedicated instance (`p`) | GCP `g2-standard-4` with one NVIDIA L4 in `us-central1`, `$0.706832276/hour` | Verified public list-rate row in `pricing.json` | Select another verified row, set `overrides.instance_id`, or maintain the catalog | +| API comparison (`a`) | `$3.75 / 1,000` inferences | `pricing.json`: 1,000 input and 250 output tokens at the catalogued public list rates | Edit **Per-call API / 1k**, set `overrides.api_price_per_1k_inferences_usd`, or maintain the catalog | +| Seconds per hour | `3,600` | Unit conversion in `app/cost.py` | Not configurable | + +The engine applies the same equations at `q_low`, `q`, and `q_high` where a range is required: + +```text +instances_needed(q) = ceil(q / t) +monthly_cost(q) = instances_needed(q) × p × h +idle_spend(q) = monthly_cost(q) × (1 − u / 100) + +monthly_inferences(q) = q × h × 3,600 × (u / 100) +cost_per_1k(q) = monthly_cost(q) / monthly_inferences(q) × 1,000 + +api_cost_at_full_utilisation(q) = q × h × 3,600 / 1,000 × a +breakeven_utilisation(q) = min(100, monthly_cost(q) / api_cost_at_full_utilisation(q) × 100) ``` -Validate the package and one grounded request: +For the default medium band, for example, `instances_needed(50) = ceil(50 / 20) = 3`; the rest can be reproduced by substituting the table values above. Workload type and model-family guesses do not enter the cost equations. + +## Quickstart + +Python 3.11 or later and Docker are required. Create `.env` from the tracked template and set `GEMINI_API_KEY` before starting the container. ```bash -python scripts/validate_environment.py -python scripts/validate_environment.py --live +git clone https://github.com/daetan999/SCAI.git +cd SCAI +cp .env.example .env +docker build -t scai . +docker run --rm --env-file .env -p 8080:8080 scai ``` -Deploy from source using the included non-root Dockerfile: +Open [http://localhost:8080](http://localhost:8080). + +The included deploy script builds from source and deploys the `scai` service to Cloud Run: ```bash +export GOOGLE_CLOUD_PROJECT="your-project-id" bash scripts/deploy_cloud_run.sh ``` -See [runbook.md](runbook.md) for the full event-day upload, validation, Web Preview, Cloud Run, batch, fallback, and screenshot procedure. +It defaults to `asia-southeast1`; set `CLOUD_RUN_REGION` to override the region. The deploying identity needs permission to build and deploy Cloud Run services and to use the configured Gemini endpoint. -## API +## Batch mode -- `POST /api/analyse` — `{"company": "...", "overrides": {...}}` -- `POST /api/estimate` — reruns Stage 2 only; no search and no model call -- `POST /api/batch` — CSV upload, one company or URL per row, maximum 50 -- `GET /api/pricing` — current `pricing.json` -- `GET /health` — `{"ok": true}` +Upload a UTF-8 CSV whose first column contains one company name or public careers-page URL per row. The optional header may be `company`, `company_name`, or `url`; a batch is limited to 50 rows and 1 MB. -Batch output columns: - -```text -company,serves_models,idle_spend_low,idle_spend_high,top_signal_url,subject,email_body,no_signal_found +```csv +company +Google Cloud +NVIDIA ``` -## Assumptions - -Every assumption is visible and editable in the cost panel. Editing the table calls only `/api/estimate`; it does not re-run Google Search. - -| Assumption | Default | Basis | -|---|---:|---| -| Utilization | 15% | Hackathon product specification. This is deliberately conservative and user-overridable. | -| Peak QPS — small | 5 | Hackathon product specification. | -| Peak QPS — medium | 50 | Hackathon product specification. | -| Peak QPS — large | 500 | Hackathon product specification. | -| Hours per month | 730 | Hackathon product specification; average month approximation. | -| Low/high range | Peak QPS −50% / +50% | Hackathon product specification. | -| Enabled instance | GCP `g2-standard-4` with one NVIDIA L4 in `us-central1` | Public on-demand list rate from the official [accelerator-optimized VM pricing page](https://cloud.google.com/products/compute/pricing/accelerator-optimized), verified 2026-08-04. | -| Throughput proxy | 4 / 20 / 100 inferences per second for small / medium / large workload bands | Editable planning placeholders for the demo, not provider benchmarks. Replace with a benchmark for the customer's model, precision, batch size, sequence length, and latency target. | -| Per-call comparison | $3.75 per 1,000 inferences | Derived from official [Gemini 3.5 Flash global standard pricing](https://cloud.google.com/gemini-enterprise-agent-platform/generative-ai/pricing): 1,000 input tokens and 250 output tokens per inference at $1.50/M input and $9.00/M output tokens. Token counts are visible modeling assumptions, not a claim about the prospect. | - -### Cost formulas +The response columns are: ```text -instances_needed = ceil(peak_qps / throughput_per_instance) -monthly_cost = instances_needed × hourly_usd × hours_per_month -idle_spend = monthly_cost × (1 − utilization) -cost_per_1k = monthly_cost / estimated_monthly_inferences × 1,000 -breakeven_utilization = dedicated_monthly_cost / API_cost_at_full_utilization +company,serves_models,idle_spend_low,idle_spend_high,top_signal_url,subject,email_body,no_signal_found ``` -The estimate excludes storage, networking, data transfer, support, taxes, committed-use discounts, reserved capacity, autoscaling behavior, engineering labor, and model-specific optimization. It is a discovery hypothesis, not a quote. - -## Pricing integrity +With the app running, submit the tracked example file and save the response: -`pricing.json` is intentionally hand-editable. A row with `hourly_usd: null` or `TODO_VERIFY: true` is disabled in the interface and rejected by the cost engine. A visible TODO is safer than a fabricated rate. +```bash +curl -fsS -F 'file=@sample_companies.csv' http://localhost:8080/api/batch -o scai-batch-results.csv +``` -Pricing verified as of `2026-08-04`. Recheck official pages before external use. +## Limitations -## Scope +- Estimates are ranges derived from public list rates. They ignore committed-use discounts, negotiated pricing, spot capacity, taxes, storage, networking, support, and engineering labour. +- Throughput figures are coarse planning bands, not benchmarks. Model architecture, precision, batch size, sequence length, hardware, and latency targets can move the result materially. +- Signal extraction depends on a company publishing something indexable. Absence of signal is not absence of workload. +- SCAI does not detect batch versus real-time inference reliably. A wrong workload classification makes the chosen throughput band less useful. +- The default assumes dedicated, continuously provisioned instances. Autoscaling, shared clusters, serverless endpoints, queues, and burst patterns are not modelled. +- Source grounding constrains citations; it does not prove that a source is current, complete, or representative of the deployed system. +- The API comparison uses a fixed token-shape assumption. It is not a quote for a prospect's actual traffic. -- Drafts email only; it never sends. -- Google Search grounding only; no direct website scraping. -- No database, authentication, user accounts, CRM, or stored request state. -- Maximum batch size: 50 rows with a three-request in-process guard. +## Disclaimer -This is a hackathon prototype. Review every source, quote, assumption, and drafted claim before using it. +SCAI is not affiliated with, endorsed by, or representing any cloud or GPU vendor. Pricing data is from public list pages and is accurate only as of the `as_of` date in `pricing.json`. diff --git a/docs/01-empty.png b/docs/01-empty.png new file mode 100644 index 0000000..77735e4 Binary files /dev/null and b/docs/01-empty.png differ diff --git a/docs/02-signals.png b/docs/02-signals.png new file mode 100644 index 0000000..90a9d2b Binary files /dev/null and b/docs/02-signals.png differ diff --git a/docs/03-cost.png b/docs/03-cost.png new file mode 100644 index 0000000..d1798e5 Binary files /dev/null and b/docs/03-cost.png differ diff --git a/docs/04-email.png b/docs/04-email.png new file mode 100644 index 0000000..a93e8be Binary files /dev/null and b/docs/04-email.png differ diff --git a/docs/05-no-signal.png b/docs/05-no-signal.png new file mode 100644 index 0000000..18a5433 Binary files /dev/null and b/docs/05-no-signal.png differ diff --git a/docs/banner.svg b/docs/banner.svg new file mode 100644 index 0000000..135511c --- /dev/null +++ b/docs/banner.svg @@ -0,0 +1,20 @@ + + SCAI + Signal and Cost engine for AI Infrastructure, with a mostly idle GPU utilisation bar. + + + + SCAI + + Signal & Cost engine for AI Infrastructure + + + + GPU UTILISATION + + + + IDLE REGION · 85% + 15% + + diff --git a/docs/demo.gif b/docs/demo.gif new file mode 100644 index 0000000..a1d36ee Binary files /dev/null and b/docs/demo.gif differ diff --git a/requirements-dev.txt b/requirements-dev.txt index 1441c92..454c480 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -1,2 +1,5 @@ -r requirements.txt pytest==9.1.1 +pytest-cov==6.3.0 +playwright==1.61.0 +Pillow==12.3.0 diff --git a/scripts/capture.py b/scripts/capture.py new file mode 100644 index 0000000..8166c99 --- /dev/null +++ b/scripts/capture.py @@ -0,0 +1,243 @@ +"""Capture reproducible README media without making Gemini API calls.""" + +from __future__ import annotations + +import argparse +import asyncio +import json +import os +import subprocess +import sys +import tempfile +import time +import urllib.request +from contextlib import contextmanager +from pathlib import Path +from typing import Iterator + +from PIL import Image +from playwright.async_api import Browser, Page, Route, async_playwright + +ROOT = Path(__file__).resolve().parents[1] +if str(ROOT) not in sys.path: + sys.path.insert(0, str(ROOT)) + +from app.schemas import AnalyseResponse + +DOCS_DIR = ROOT / "docs" +FIXTURE_PATH = ROOT / "scripts" / "fixtures" / "capture.json" +BASE_URL = "http://127.0.0.1:8765" +VIEWPORT = {"width": 1440, "height": 900} +DEVICE_SCALE_FACTOR = 2 +MAX_GIF_BYTES = 5_000_000 + + +def load_fixture(path: Path = FIXTURE_PATH) -> dict[str, dict]: + """Load both recorded API states and validate them against the public schema.""" + payload = json.loads(path.read_text(encoding="utf-8")) + return { + key: AnalyseResponse.model_validate(payload[key]).model_dump( + mode="json", by_alias=True + ) + for key in ("success", "no_signal") + } + + +def _wait_for_server(url: str, timeout_seconds: float = 15.0) -> None: + deadline = time.monotonic() + timeout_seconds + while time.monotonic() < deadline: + try: + with urllib.request.urlopen(url, timeout=1) as response: + if response.status == 200: + return + except OSError: + time.sleep(0.1) + raise RuntimeError(f"Timed out waiting for {url}") + + +@contextmanager +def run_server() -> Iterator[None]: + """Run the real FastAPI app; Playwright replaces only the model-backed route.""" + environment = {**os.environ, "APP_MODE": "fixture"} + process = subprocess.Popen( + [ + sys.executable, + "-m", + "uvicorn", + "app.main:app", + "--host", + "127.0.0.1", + "--port", + "8765", + ], + cwd=ROOT, + env=environment, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + ) + try: + _wait_for_server(f"{BASE_URL}/health") + yield + finally: + process.terminate() + try: + process.wait(timeout=5) + except subprocess.TimeoutExpired: + process.kill() + process.wait(timeout=5) + + +async def _install_fixture_route(page: Page, fixture: dict[str, dict]) -> None: + async def fulfil_analysis(route: Route) -> None: + request_payload = json.loads(route.request.post_data or "{}") + fixture_name = ( + "no_signal" + if request_payload.get("company") == "No Signal Labs" + else "success" + ) + await asyncio.sleep(0.75) + await route.fulfill(json=fixture[fixture_name]) + + await page.route("**/api/analyse", fulfil_analysis) + + +async def _new_page(browser: Browser, fixture: dict[str, dict]) -> Page: + context = await browser.new_context( + viewport=VIEWPORT, + device_scale_factor=DEVICE_SCALE_FACTOR, + color_scheme="light", + reduced_motion="reduce", + ) + page = await context.new_page() + await _install_fixture_route(page, fixture) + await page.goto(BASE_URL, wait_until="networkidle") + await page.add_style_tag( + content="*, *::before, *::after { cursor: none !important; " + "scroll-behavior: auto !important; }" + ) + await page.locator("#analyse-button").wait_for(state="visible") + await page.wait_for_function( + "document.querySelector('#analyse-button')?.disabled === false" + ) + return page + + +async def _capture_viewport(page: Page, path: Path) -> None: + await page.screenshot(path=path, full_page=False, animations="disabled") + + +async def _scroll_to(page: Page, selector: str) -> None: + await page.eval_on_selector( + selector, + "element => window.scrollTo(0, element.getBoundingClientRect().top " + "+ window.scrollY - 24)", + ) + await page.wait_for_timeout(100) + + +def _write_gif(frame_paths: list[Path], output_path: Path) -> None: + durations_ms = [1200, 1100, 1900, 1900, 2100, 2100] + frames: list[Image.Image] = [] + try: + for path in frame_paths: + with Image.open(path) as source: + resized = source.convert("RGB").resize( + (960, 600), Image.Resampling.LANCZOS + ) + frames.append( + resized.quantize( + colors=64, + method=Image.Quantize.MEDIANCUT, + dither=Image.Dither.NONE, + ) + ) + frames[0].save( + output_path, + save_all=True, + append_images=frames[1:], + duration=durations_ms, + loop=0, + optimize=True, + disposal=2, + ) + finally: + for frame in frames: + frame.close() + if output_path.stat().st_size >= MAX_GIF_BYTES: + raise RuntimeError( + f"{output_path.name} is {output_path.stat().st_size} bytes; " + f"the limit is {MAX_GIF_BYTES} bytes" + ) + + +async def capture() -> None: + fixture = load_fixture() + DOCS_DIR.mkdir(parents=True, exist_ok=True) + with tempfile.TemporaryDirectory(prefix="scai-capture-") as temporary: + temporary_dir = Path(temporary) + async with async_playwright() as playwright: + browser = await playwright.chromium.launch() + try: + page = await _new_page(browser, fixture) + await _capture_viewport(page, DOCS_DIR / "01-empty.png") + demo_frames = [temporary_dir / "01-empty.png"] + await _capture_viewport(page, demo_frames[0]) + + await page.locator("#company").fill("Example Robotics") + await page.locator("#analyse-button").click() + await page.locator("#signals-body .loading-card").wait_for() + demo_frames.append(temporary_dir / "02-loading.png") + await _capture_viewport(page, demo_frames[-1]) + await page.locator("#email-body .email-block").wait_for() + + await page.locator("#signals-panel").screenshot( + path=DOCS_DIR / "02-signals.png", animations="disabled" + ) + await page.locator("#cost-panel").screenshot( + path=DOCS_DIR / "03-cost.png", animations="disabled" + ) + await page.locator("#email-panel").screenshot( + path=DOCS_DIR / "04-email.png", animations="disabled" + ) + + await page.evaluate("window.scrollTo(0, 0)") + demo_frames.append(temporary_dir / "03-complete.png") + await _capture_viewport(page, demo_frames[-1]) + for index, selector in enumerate( + ("#signals-panel", "#cost-panel", "#email-panel"), start=4 + ): + await _scroll_to(page, selector) + demo_frames.append(temporary_dir / f"{index:02d}-panel.png") + await _capture_viewport(page, demo_frames[-1]) + + refusal_page = await _new_page(browser, fixture) + await refusal_page.locator("#company").fill("No Signal Labs") + await refusal_page.locator("#analyse-button").click() + await refusal_page.locator("#signals-body .no-signal").wait_for() + await refusal_page.locator("#signals-panel").screenshot( + path=DOCS_DIR / "05-no-signal.png", animations="disabled" + ) + await refusal_page.context.close() + + _write_gif(demo_frames, DOCS_DIR / "demo.gif") + await page.context.close() + finally: + await browser.close() + + +def main() -> int: + parser = argparse.ArgumentParser( + description="Capture README screenshots and a fixture-backed demo GIF." + ) + parser.parse_args() + with run_server(): + asyncio.run(capture()) + for path in sorted(DOCS_DIR.glob("0[1-5]-*.png")): + print(f"Captured {path.relative_to(ROOT)}") + demo = DOCS_DIR / "demo.gif" + print(f"Captured {demo.relative_to(ROOT)} ({demo.stat().st_size} bytes)") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/fixtures/capture.json b/scripts/fixtures/capture.json new file mode 100644 index 0000000..2359d38 --- /dev/null +++ b/scripts/fixtures/capture.json @@ -0,0 +1,93 @@ +{ + "success": { + "signals": { + "company": "Example Robotics", + "as_of": "2026-08-04", + "no_signal_found": false, + "signals": [ + { + "type": "hiring", + "summary": "Hiring for low-latency model serving infrastructure", + "evidence_quote": "Own the production inference platform, including GPU scheduling, latency, and reliability.", + "source_url": "https://example.com/careers/ml-platform", + "date": "2026-07-28", + "confidence": "high" + }, + { + "type": "engineering_blog", + "summary": "Engineering team describes a real-time recommendation service", + "evidence_quote": "Recommendations are generated online for each customer request.", + "source_url": "https://example.com/engineering/recommendations", + "date": "2026-06-12", + "confidence": "medium" + } + ], + "inferred": { + "serves_models_in_production": true, + "workload_type": "realtime", + "model_family_guess": null, + "scale_band": "medium", + "reasoning": "A model-serving role and online recommendation architecture indicate a production real-time inference workload.", + "confidence": "medium" + } + }, + "cost": { + "assumptions": [ + {"name": "Scale band", "value": "medium", "basis": "signal", "source_url": "https://example.com/careers/ml-platform"}, + {"name": "Peak QPS", "value": "50", "basis": "default", "source_url": null}, + {"name": "Assumed utilisation", "value": "15%", "basis": "default", "source_url": null}, + {"name": "Hours per month", "value": "730", "basis": "default", "source_url": null}, + {"name": "Throughput per instance", "value": "20 inferences/sec", "basis": "default", "source_url": "https://cloud.google.com/products/compute/pricing/accelerator-optimized"}, + {"name": "Dedicated instance", "value": "NVIDIA L4 (g2-standard-4, us-central1) at $0.706832/hour", "basis": "default", "source_url": "https://cloud.google.com/products/compute/pricing/accelerator-optimized"}, + {"name": "Per-call API comparison", "value": "$3.75 per 1,000 inferences", "basis": "default", "source_url": "https://cloud.google.com/gemini-enterprise-agent-platform/generative-ai/pricing"} + ], + "pricing_rows_used": [ + { + "id": "gcp-g2-standard-4-l4", + "provider": "GCP", + "accelerator": "NVIDIA L4 (g2-standard-4, us-central1)", + "hourly_usd": 0.706832276, + "source_url": "https://cloud.google.com/products/compute/pricing/accelerator-optimized", + "est_throughput_inferences_per_sec": {"small": 4.0, "medium": 20.0, "large": 100.0}, + "TODO_VERIFY": false + } + ], + "monthly_cost_usd_low": 1031.97512296, + "monthly_cost_usd_high": 2063.95024592, + "assumed_utilisation_pct": 15.0, + "idle_spend_monthly_usd": 1315.768281774, + "idle_spend_monthly_usd_low": 877.1788545159999, + "idle_spend_monthly_usd_high": 1754.3577090319998, + "cost_per_1k_inferences_usd": 0.07853691955555556, + "breakeven_utilisation_pct": 0.3141476782222222, + "confidence": "medium" + }, + "email": { + "subject": "The idle range behind Example Robotics' model-serving role", + "body": "$877–$1,754 per month is the modeled idle-spend range behind Example Robotics' public ML platform signal. Your role mentions owning production inference, GPU scheduling, latency, and reliability. I mapped that signal to visible list-price and throughput assumptions; the arithmetic is deterministic and editable. If useful, I can share a free 30-minute inference cost teardown, nothing to buy.\n\n[YOUR NAME] / [YOUR LINK]", + "word_count": 58, + "signal_referenced": { + "summary": "Hiring for low-latency model serving infrastructure", + "source_url": "https://example.com/careers/ml-platform" + } + } + }, + "no_signal": { + "signals": { + "company": "No Signal Labs", + "as_of": "2026-08-04", + "no_signal_found": true, + "signals": [], + "inferred": { + "serves_models_in_production": false, + "workload_type": "unknown", + "model_family_guess": null, + "scale_band": "unknown", + "reasoning": "No usable grounded public evidence was returned.", + "confidence": "low" + } + }, + "cost": null, + "email": null + } +} diff --git a/static/index.html b/static/index.html index 6ccc534..1255291 100644 --- a/static/index.html +++ b/static/index.html @@ -220,7 +220,10 @@

${escapeHtml(signal.summary)}

} function assumptionByName(name) { return state.cost.assumptions.find((item) => item.name === name); } - function numericAssumption(name) { return parseFloat(assumptionByName(name)?.value.replace(/[^0-9.]/g, "")); } + function numericAssumption(name) { + const match = assumptionByName(name)?.value.match(/-?\d[\d,]*(?:\.\d+)?/); + return match ? Number(match[0].replaceAll(",", "")) : Number.NaN; + } function syncOverrides() { state.overrides = { diff --git a/tests/test_capture.py b/tests/test_capture.py new file mode 100644 index 0000000..7181eb0 --- /dev/null +++ b/tests/test_capture.py @@ -0,0 +1,45 @@ +from app.schemas import AnalyseResponse +from playwright.sync_api import Route, sync_playwright + +from scripts.capture import BASE_URL, load_fixture, run_server + + +def test_capture_fixture_has_complete_success_and_refusal_states(): + fixture = load_fixture() + + success = AnalyseResponse.model_validate(fixture["success"]) + refusal = AnalyseResponse.model_validate(fixture["no_signal"]) + + assert success.signals.no_signal_found is False + assert success.signals.signals + assert success.cost is not None + assert success.email is not None + + assert refusal.signals.no_signal_found is True + assert refusal.signals.signals == [] + assert refusal.cost is None + assert refusal.email is None + + +def test_cost_table_keeps_the_api_price_separate_from_its_unit(): + fixture = load_fixture() + + def fulfil_analysis(route: Route) -> None: + route.fulfill(json=fixture["success"]) + + with run_server(), sync_playwright() as playwright: + browser = playwright.chromium.launch() + page = browser.new_page() + page.route("**/api/analyse", fulfil_analysis) + page.goto(BASE_URL, wait_until="networkidle") + page.locator("#company").fill("Example Robotics") + page.locator("#analyse-button").click() + page.locator("#email-body .email-block").wait_for() + + api_price = page.locator( + '#cost-body [data-field="api_price_per_1k_inferences_usd"]' + ).input_value() + + browser.close() + + assert api_price == "3.75"