Skip to content

test: use the current triage VM slug, not the deprecated win10 one - #317

Merged
sbneto merged 1 commit into
developfrom
fix-deprecated-triage-vm-slug
Jul 30, 2026
Merged

test: use the current triage VM slug, not the deprecated win10 one#317
sbneto merged 1 commit into
developfrom
fix-deprecated-triage-vm-slug

Conversation

@sbneto

@sbneto sbneto commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

TL;DR

The sandbox tests dispatched 'triage' with vm_slug='win10-build-15063' at 8 call sites. That isn't a valid triage VM — the server rewrites any slug containing win10 to windows11-21h2-x64 for backward compatibility and logs an ERROR each time. So every live e2e run carried 8 self-inflicted server-side ERROR lines.

  • 8 sites → 'windows11-21h2-x64' (4 in client_scan_test.py, 4 in async_client_test.py).
  • triage offers windows11-21h2-x64, ubuntu-22.04-amd64, android-11-x64; the win10-style slug belongs to a different provider.

Cassettes are deliberately untouched

vm_slug travels in the POST body, and the VCR matcher is [method, scheme, host, port, path, query] — the body isn't matched, so replay is unaffected. Hand-editing cassettes is also against the recording convention (they're produced by running against a live stack). The 8 cassettes that still carry the old recorded body will pick up the new slug naturally the next time they're re-recorded delete-driven.

The server's response was already the rewritten one, so the recorded responses stay correct either way.

Verification

pytest test/client_scan_test.py test/async_client_test.py -k "sandbox or sample"13 passed on replay.

The sandbox tests dispatched 'triage' with vm_slug='win10-build-15063' at 8 call
sites. That is not a valid triage VM: the server rewrites any slug containing
'win10' to 'windows11-21h2-x64' for backward compatibility and logs an ERROR each
time it does. Every e2e run therefore carried 8 server-side ERROR lines that were
entirely self-inflicted.

Send the real slug instead. 'triage' offers windows11-21h2-x64 (plus
ubuntu-22.04-amd64 and android-11-x64); the win10-style slug belongs to a
different provider and does not apply here.

Cassettes are deliberately NOT touched. vm_slug travels in the POST body and the
VCR matcher is [method, scheme, host, port, path, query], so replay is unaffected
— and hand-editing cassettes is against the recording convention. The old recorded
bodies simply get refreshed the next time these cassettes are re-recorded
delete-driven against a live stack. Verified: 13 sandbox/sample tests pass on
replay.
@claude

claude Bot commented Jul 30, 2026

Copy link
Copy Markdown

Reviewed against AGENTS.md + specs/04-testing.md. Clean — no action needed.

Verified:

  • All 8 triage dispatch sites updated; grep -rn 'win10\|15063' test/ src/ *.md returns nothing outside the 8 cassettes' recorded request bodies. Remaining win-10-build-19041 uses are all cape, which is correct.
  • windows11-21h2-x64 is genuinely advertised by the triage provider (test/vcr/test_sandbox_providers.vcr lists windows11-21h2-x64, ubuntu-22.04-amd64, android-11-x64).
  • Leaving cassettes untouched is right and matches invariant 3 ("never hand-edited"). Replay is unaffected: vm_slug rides in the POST body, and neither module matches on body — async_client_test.py pins match_on=['method','scheme','host','port','path','query'] and client_scan_test.py uses vcrpy's defaults, which are the same set. Each of the 8 cassettes carries the old slug exactly once (the request body); every recorded response already contains the rewritten windows11-21h2-x64, so no assertion shifts on replay.
  • Gitflow: base is develop, no pyproject.toml version bump, test: prefix, no ticket IDs or private repo names. Test-only change — no public surface touched, so no bump decision required.
  • No spec update needed; specs/04-testing.md's e2e conventions cover engines/samples, not VM slugs.

@sbneto
sbneto merged commit cd1590c into develop Jul 30, 2026
2 checks passed
@sbneto
sbneto deleted the fix-deprecated-triage-vm-slug branch July 30, 2026 17:29
This was referenced Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants