[CI Postmortem] 'Verify ssh connection' failing on Windows 'Dotnet tests' job

[rolfbjarne]

Infrastructure Failure Report (automated)

Failing Step: Verify ssh connection
Period: Apr 27 – May 4, 2026
Occurrences: 2 across 2 PRs (#25197, #25261)
Affected Job: Dotnet tests (Windows)

Error Message

PowerShell exited with code '1'.

Occurrences

PR	Build	Bot	Direct Link
#25261	13954991	VSM-XAM-126	Verify ssh connection
#25197	13955255	VSM-XAM-141	Verify ssh connection

Impact

When the ssh connection verification fails, the entire Windows 'Dotnet tests' job fails. Downstream steps like 'Prepare tests results and Html Report' then also fail (cascade).

---

This issue was automatically generated by CI post-mortem analysis.
Originally tracked 'Prepare tests results' failures — corrected after root cause re-analysis (see comment).

[rolfbjarne]

Correction: Root Cause Re-Analysis

The 'Prepare tests results and Html Report' step is a cascading failure, not the root cause. Analyzing the first truly failed step in each 'Dotnet tests' job:

Build	PR	Actual First Failure	Error
13954991	#25261	Verify ssh connection	PowerShell exited with code '1'
13955255	#25197	Verify ssh connection	PowerShell exited with code '1'
13955798	#25269	Checkout	Git fetch failed with exit code 128
13958912	#25279	Checkout	Git fetch failed with exit code 128

This means 2 occurrences are 'Verify ssh connection' failures and 2 are git checkout failures — different root causes. Renaming this issue to cover the ssh connection failures. The git checkout failures in builds 13955798/13958912 are unrelated infrastructure issues.

[rolfbjarne]

macOS Bot Correlation

Both 'Verify ssh connection' failures on the Windows bots were connecting to the same macOS bot: VSM-XAM-13.Tahoe.arm64.

Build	Windows Bot	macOS Bot (from 'Reserve macOS bot for tests')
13954991	VSM-XAM-126	VSM-XAM-13.Tahoe.arm64
13955255	VSM-XAM-141	VSM-XAM-13.Tahoe.arm64

This is the same bot causing failures in #25314 (Download secrets: azdev-secrets). The ssh connection failures are a symptom of VSM-XAM-13 having network/connectivity issues — not a problem with the Windows bots.

[rolfbjarne]

I've rebooted VSM-XAM-13, so this should be fixed for now.