feat: typed exception for a withheld known-good binary

specs/00-overview.md

@@ -19,7 +19,7 @@ What the `polyswarm-api` Python SDK is, what it ships, where it sits in the plat
 | `polyswarm_api.PolyswarmSession` / `polyswarm_api.aio.AsyncPolyswarmSession` | Transport classes. Own the underlying `httpx.{,Async}Client`, expose `execute(request)` and `upload_file(url, artifact, …)`. Subclass and inject to customize transport behaviour. |
 | `polyswarm_api.resources` | Per-domain resource classes (`ArtifactInstance`, `LocalArtifact`, `HistoricalHunt`, `LiveYaraRuleset`, `YaraRuleset`, `MetadataFieldProperties`, `LLMPromptConfig`, …). Wrappers over the server's JSON responses. Builder classmethods (`create` / `get` / `update` / `delete` / `list` / etc.) return `PolyswarmRequest` descriptors. |
 | `polyswarm_api.core.PolyswarmRequest` | Pure description of an HTTP call (method, URL, params, body, parser). Constructed by resource builders; handed to a session for execution. No I/O on the descriptor itself. |
-| `polyswarm_api.exceptions` | Exception hierarchy (`PolyswarmException` → `RequestException`, `NotFoundException`, `FailedInstanceException`, `NoResultsException`, `UsageLimitsExceededException`, `InvalidValueException`, `TimeoutException`). |
+| `polyswarm_api.exceptions` | Exception hierarchy (`PolyswarmException` → `RequestException`, `NotFoundException` → `KnownGoodWithheldException`, `FailedInstanceException`, `NoResultsException`, `UsageLimitsExceededException`, `InvalidValueException`, `TimeoutException`). |
 
 ## Where it sits
 

specs/01-architecture.md

@@ -212,7 +212,7 @@ A non-`BaseJsonResource` parser is a **download** and instead takes the streamin
 `parse_response(response, request)`:
 
 - HEAD: `request._result = response.status_code`; return.
-- Non-2xx: extract JSON body into `request.json` / `request.status` / `request.errors`; dispatch on status code to typed exception class (`NotFoundException`, `FailedInstanceException`, `UsageLimitsExceededException`, `RequestException`); raise.
+- Non-2xx: extract JSON body into `request.json` / `request.status` / `request.errors`; dispatch on status code to typed exception class (`NotFoundException`, `FailedInstanceException`, `UsageLimitsExceededException`, `RequestException`); raise. The 404 arm also reads the extracted `errors` payload: a dict carrying `code == 'KNOWN_GOOD'` raises the `NotFoundException` subclass `KnownGoodWithheldException` instead.
 - 2xx without `result_parser`: return (fire-and-forget endpoints like `notification_webhook_test`).
 - 2xx with `BaseJsonResource` parser: extract JSON, populate pagination metadata (`total`, `limit`, `offset`, `has_more`, `_paginated`), dispatch on `result_parser.parse_result_list` (list) or `.parse_result` (single).
 - 2xx with non-`BaseJsonResource` parser: pass `(api, response)` directly to `result_parser.parse_result` (used for `LocalArtifact` file downloads).
@@ -267,11 +267,11 @@ Both paths are covered by respx tests (`test_async_pagination_*` in `async_clien
 
 Every HTTP-level error maps to a subclass of `PolyswarmException`:
 
-- 404 → `NotFoundException`
+- 404 → `NotFoundException` — or `KnownGoodWithheldException` (its subclass) when the body's `errors` dict carries `code == 'KNOWN_GOOD'`: the artifact is a known-good binary whose bytes are never stored or served. Subclassing keeps every existing `except NotFoundException` caller working; the exception exposes the flagging feeds as `.sources`.
 - 422 → `FailedInstanceException`
 - 429 → `UsageLimitsExceededException`
 - 204 on a request that expects data (JSON-parser GET **or** streaming download) → `NoResultsException` — the server did the work but matched nothing. **HEAD is exempt**: it returns the raw status code as the result (so `exists()` reads a 204 as "known-absent" rather than raising).
-- Other non-2xx → `RequestException`
+- Other non-2xx → `RequestException`. Its message renders the request diagnostics plus the envelope's `errors` slot, in **either** shape: the legacy **list** (one entry per line) or the way-forward **mapping** (`key=value` lines). The mapping shape is not 404-specific — the server can send it on any status — and iterating a mapping yields only its keys, so it must not be rendered like a list or every value is silently dropped from the message. (A bare string is rendered as-is for the same reason: iterating one yields characters.) Note the render itself reaches only **this** arm: `_bad_status_message` has a single call site, so the 404 / 422 / 429 arms build their messages from `request._result` alone and a mapping-shaped `errors` does not appear in them. The payload is still reachable at `exc.request.errors` on every arm.
 - Client-side validation failures (bad hash, missing kwarg) → `InvalidValueException`
 - Polling timeouts → `TimeoutException`
 

specs/02-resources.md

@@ -140,7 +140,7 @@ Behaviour:
 | Branch | Action |
 |---|---|
 | `request.method == 'HEAD'` | `request._result = response.status_code`; return. |
-| Non-2xx | Extract JSON body into `request.json` / `.status` / `.errors`. Dispatch on status code: 404 → `NotFoundException`, 422 → `FailedInstanceException`, 429 → `UsageLimitsExceededException`, else → `RequestException`. Raise. |
+| Non-2xx | Extract JSON body into `request.json` / `.status` / `.errors`. Dispatch on status code: 404 → `NotFoundException` (→ `KnownGoodWithheldException` when `errors['code'] == 'KNOWN_GOOD'`), 422 → `FailedInstanceException`, 429 → `UsageLimitsExceededException`, else → `RequestException`. Raise. |
 | 2xx, no `result_parser` | Return (fire-and-forget endpoints). |
 | 2xx, `result_parser` is `BaseJsonResource` subclass, status 204 | Raise `NoResultsException`. |
 | 2xx, `BaseJsonResource` parser | Extract JSON. Populate pagination metadata (`_paginated` / `total` / `limit` / `offset` / `has_more`). Find `result` or `results` key in body. Dispatch on `result_parser.parse_result_list` (list) or `.parse_result` (single). |
@@ -302,9 +302,19 @@ it parses to `None` with no behaviour change). It lets a consumer recognise a
 known-good-bypassed scan via `state == 'KNOWN_GOOD'` even when `known_good` above is
 `None` (the sha matched no `KnownGood`). The raw numeric `bounty_state` is unchanged.
 
+`state == 'KNOWN_GOOD'` (equivalently the status the server reports for the instance)
+is **the** signal for the *typed refusal* — the platform never stores or serves a
+known-good binary, there is deliberately no separate "withheld" field to read, and a
+download attempted anyway raises `KnownGoodWithheldException` (see §"Exceptions thrown
+by parsing"); the metadata — the flagging feeds plus any scan data already collected —
+stays readable. It is not the signal for "bytes are unavailable" in general:
+`state == 'NOT_STORED'` (below) also has no bytes — nothing was ever stored for that
+instance — but its download 404s **plainly**, without the `KNOWN_GOOD` code, because
+nothing is being withheld by policy any more.
+
 Classmethod builders (each returns a `PolyswarmRequest` descriptor):
 
-- `exists_hash(api, hash_value, hash_type, require_scan=False)` — HEAD request, returns the status code as the result.
+- `exists_hash(api, hash_value, hash_type, require_scan=False)` — HEAD request, returns the status code as the result. `require_scan=True` narrows the answer to artifacts that were actually scanned. Being catalogued as known-good is **not** a scan — a hash whose only record is a known-good reference answers present without `require_scan` (the reference is a real searchable record) and absent with it.
 - `search_hash(api, hash_value, hash_type)` — GET `/search/hash/{hash_type}`.
 - `search_url(api, url)` — GET `/search/url`.
 - `list_scans(api, hash_value)` — GET `/search/instances`.
@@ -381,9 +391,10 @@ This keeps the body off the heap for `folder`/file-handle destinations — parit
 
 - `NoResultsException` — HTTP 204 with a typed `result_parser`.
 - `NotFoundException` — HTTP 404, or a JSON-decode failure on a 404.
+- `KnownGoodWithheldException` (a `NotFoundException` subclass) — HTTP 404 whose `errors` payload is a dict with `code == 'KNOWN_GOOD'`: the artifact is a known-good binary and its bytes are withheld by design. Carries `.sources` (the flagging known-good feeds, always a list of strings — normalised in the exception's constructor — and `[]` when none were named or the payload arrived in another shape). Any other 404 — a different code, a legacy list-shaped `errors`, or no `errors` at all — stays a plain `NotFoundException`.
 - `FailedInstanceException` — HTTP 422.
 - `UsageLimitsExceededException` — HTTP 429.
-- `RequestException` — any other non-2xx.
+- `RequestException` — any other non-2xx. Its message appends the envelope's `errors` slot rendered for whichever shape arrived: a **list** renders one entry per line (the legacy shape), a **mapping** renders `key=value` lines (the way-forward shape, which the server can send on any status — not just the 404 the `code` contract was introduced for), and anything else renders as a plain string. This applies to the `RequestException` arm only — see [`01-architecture.md`](./01-architecture.md).
 
 Each is raised by `parse_response`. `RequestException.__init__` attaches the descriptor as `.request`, so callers downstream read `exc.request.status_code`, `exc.request.json`, etc. The session does not catch and rewrap — attachment happens at construction time.
 

specs/03-endpoints.md

@@ -25,7 +25,7 @@ The full catalogue of methods on the public client surface and which transport h
 
 | Method | Resource builder | Notes |
 |---|---|---|
-| `exists(hash_, hash_type=None, require_scan=False)` | `ArtifactInstance.exists_hash` | HEAD; `bool` from status code — `True` **only** for `200` (present). `204` means "absent" (the request succeeded but matched no artifact) and `404` also maps to absent, so both are `False`. Do **not** treat this as a generic `2xx` check: `204` is a successful status that means the opposite of "exists". |
+| `exists(hash_, hash_type=None, require_scan=False)` | `ArtifactInstance.exists_hash` | HEAD; `bool` from status code — `True` **only** for `200` (present). `204` means "absent" (the request succeeded but matched no artifact) and `404` also maps to absent, so both are `False`. Do **not** treat this as a generic `2xx` check: `204` is a successful status that means the opposite of "exists". `require_scan=True` narrows *found* to artifacts that were actually **scanned**: being catalogued as known-good is not a scan, so a hash whose only record is a known-good reference reports absent under `require_scan` and present without it. **The server's codes here are a frozen contract** — see artifact-index `specs/09-hash-search-head-contract.md`; neither side may widen what counts as found, because a widening moves a caller's case from `False` to `True` with no error and no log line (that is the 4.0 `exists()` inversion, shipped in 4.0.0/4.1.0 and fixed in 4.2.0). |
 | `lookup(scan)` | `ArtifactInstance.lookup_uuid` | |
 | `rescan(hash_, hash_type=None, scan_config=None)` | `ArtifactInstance.rescan` | |
 | `rescan_id(scan, scan_config=None)` | `ArtifactInstance.rescan_id` | |
@@ -106,11 +106,27 @@ Internal-only CRUD for the `/known-good` binary resource (distinct from the IOC
 
 | Method | Resource builder | Notes |
 |---|---|---|
-| `download(out_dir, hash_, hash_type=None)` | `LocalArtifact.download` | Closes the handle before returning. |
+| `download(out_dir, hash_, hash_type=None)` | `LocalArtifact.download` | Closes the handle before returning. Raises `KnownGoodWithheldException` (see below). |
 | `download_id(out_dir, instance_id)` | `LocalArtifact.download_id` | Same. |
-| `download_sandbox_artifact(out_dir, sandbox_task_id, instance_id)` | `LocalArtifact.download_sandbox_artifact` | Same. |
-| `download_archive(out_dir, s3_path)` | `LocalArtifact.download_archive` | Same. |
-| `download_to_handle(hash_, fh, hash_type=None)` | `LocalArtifact.download` | Streams to an existing file handle. |
+| `download_sandbox_artifact(out_dir, sandbox_task_id, instance_id)` | `LocalArtifact.download_sandbox_artifact` | Same — and the gate applies to **every** sandbox artifact by its own sha256 (dropped file, screenshot, report, …); the server-side model has no sample-vs-evidence carve-out. |
+| `download_archive(out_dir, s3_path)` | `LocalArtifact.download_archive` | Closes the handle before returning. **Not** an artifact-index call (see below), so it never raises `KnownGoodWithheldException`. |
+| `download_to_handle(hash_, fh, hash_type=None)` | `LocalArtifact.download` | Streams to an existing file handle. Same refusal. |
+
+**Every artifact-index-served download** (`download`, `download_id`, `download_to_handle`,
+`download_sandbox_artifact`) **can refuse with `KnownGoodWithheldException`** — the sha256 is
+catalogued as a known-good binary, so its bytes are withheld by design rather than missing. It
+subclasses `NotFoundException` (raised from the shared `_raise_for_status` 404 arm), so existing
+`except NotFoundException` handling still catches it; catch it specifically to tell a deliberate
+refusal apart from a gone artifact, and read `.sources` for the feeds that flagged the hash.
+**Nothing is written to the destination** — `_execute_download` checks the status before it opens
+the file, so a refusal never leaves a zero-byte file behind, which to a caller would be
+indistinguishable from a download that worked.
+
+`download_archive` is the exception because it is not an artifact-index request at all: it fetches
+the caller-supplied pre-signed object-store URL from the `stream()` feed, with the `Authorization`
+header suppressed. An error there is the store's own (XML body, no coded JSON envelope), so it
+surfaces as a plain `NotFoundException` / server error from the generic arms, never as the typed
+refusal.
 
 ### Sandbox