Prepare SSH key before provisioning VM instances in test runner

[samuelvl]

What does this PR do?

Fixes a bug where integration tests running on cloud VM instances failed because the SSH key used to reach those instances wasn't ready yet when the VM was being provisioned. The provisioning step now waits for the SSH key to exist first, so cloud-based test runs work again.

This was introduced in #15422 and detected in #15449 (comment)

[infra-vault-gh-plugin-prod]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[infra-vault-gh-plugin-prod]

💔 Build Failed

• Buildkite Build
• Commit: bd4815d

Failed CI Steps

• windows:amd64:tier1:sudo:fleet
• linux:amd64:tier1:sudo:fleet
• linux:amd64:tier1:sudo:fleet

History

cc @samuelvl

[github-actions]

TL;DR

The available failed job log shows a definitive infrastructure failure during ESS teardown: oblt-cli could not authenticate to https://github.com/elastic/observability-test-environments.git, so cluster destruction failed with exit status 128 (log lines 102-112). This is unrelated to the SSH-key ordering change in pkg/testing/runner/runner.go.

The same job also reports TestMonitoringLivenessReloadable and one other integration failure, but the supplied log contains only cleanup/diagnostic output and no assertion, timeout, or component error, so that failure cannot be reliably attributed to this PR from the available evidence.

Remediation

• Fix or restore the Buildkite/oblt-cli GitHub authentication used for observability-test-environments, then rerun the failed fleet steps.
• If TestMonitoringLivenessReloadable still fails after the infrastructure issue is resolved, inspect its uploaded diagnostics and full test report for the missing primary error before changing the runner code.

Investigation details

Root Cause

The failed job linux:amd64:tier1:sudo:fleet reached test cleanup, then ESS teardown failed because Git rejected the credentials: Invalid username or token. Password authentication is not supported for Git operations. The failure is in the external oblt-cli cluster Git wrapper, not in the changed runner path.

Evidence

• Build: https://buildkite.com/elastic/elastic-agent/builds/42785
• Job/step: linux:amd64:tier1:sudo:fleet
• Log: lines 102-112 show authentication failure and the warning that cluster hosted-ess-ea-it-rjlql could not be destroyed.
• PR diff: the change only moves VM preparation (SSH key and repository archive creation) before Provision; no authentication or teardown code is modified.

Verification

• Reviewed the supplied Buildkite log, PR diff, existing PR comments, and repository code around Runner.Run and prepare.
• No local integration test was run; the failure requires Buildkite VM/ESS infrastructure.

Follow-up

The log summary DONE 183 tests, 124 skipped, 2 failures is insufficient to diagnose the second failure because its primary test output is absent from the supplied log.

---

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.