Run osquerybeat as a receiver by default and route live query actions to it via the diagnostics extension

[cmacknz]

• Closes [beats receivers] Enable osquerybeat to run as a receiver by default #14555
• Closes [9.5.0] OSQuery live queries do not work in the OTel runtime #15410
• Depends Wire the OTel manager's RegisterAction method to the extension satisfying the new ActionExtension interface beats#51872

What does this PR do?

This fixes #15410, using the approached discussed in that issue. Uses the already wired up diagnostics extension to route Fleet actions to osquerybeat receiver via HTTP over unix socket. Note that Fleet actions currently can only be routed to a component, so when an input is split into multiple per component streams only one of them could register an action handler. In OSQuery's case, it already uses the single_receiver: true option in it's spec file to avoid being split by stream so there is no effect on it, but the code handles this situation by failing loudly anyway.

The diagnosticsextension could be renamed to actionsextension at this point, but the expectation is that the action processing part of this change can eventually be removed in favor of OpAMP custom messages when we switch to using OpAMP to supervise the collector. The diagnostics extension would remain as a way for users to trigger diagnostics manually in that case.

Additionally updates the osquery monitoring integration test to perform a live query and ensure it succeeds.

This PR requires elastic/beats#51872 to merge first so that the beats submodule can be pointed to elastic/beats instead of cmacknz/beats.

How to test this PR locally

Run the TestOsqueryManager integration test.

Screenshots

Here is a successfully executed live query from a manual test:

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b cmacknz/fix-osquery-live-actions upstream/cmacknz/fix-osquery-live-actions
git merge upstream/main
git push upstream cmacknz/fix-osquery-live-actions

[infra-vault-gh-plugin-prod]

⏳ Build in-progress, with failures

• Buildkite Build
• Commit: 56d5024

Failed CI Steps

• Unit tests - fips140=only Windows 2022
• windows:amd64:tier2:sudo:default:platform-ingest-elastic-agent-windows-2016-1782867684
• windows:amd64:tier2:sudo:default:platform-ingest-elastic-agent-windows-2016-1782867684
• windows:amd64:tier2:sudo:fleet:platform-ingest-elastic-agent-windows-2016-1782867684

History

• 💔 Build #42793 failed 2f434e0

cc @cmacknz

[github-actions]

TL;DR

Build 42800 reached the Windows ESS integration suite and failed 17 tests. The captured failure is a package-version consistency assertion in TestInstallServersWithBasePath/check_agent_package_version; it is not evidence of an osquery live-action regression. The supplied log is truncated and does not include the actual version --yaml --binary-only value or the other 16 failures, so the final underlying cause cannot be determined from this artifact alone.

Remediation

• Retrieve and inspect build/TEST-report.html or the uploaded integration XML/JSON for all 17 failure details, especially the assertion diff showing the actual binary version.
• Re-run the Windows ESS job after confirming whether the installed package.version (9.5.0) disagrees with the packaged binary version; if reproducible, fix the packaging/version injection so both values match.
• Treat any failures from the new osquery live-query subtest separately; none are present in the captured excerpt.

Investigation details

Root Cause

Confirmed classification: test failure in the package-version check. The log reports:

pkgversion_common_test.go:37: package version files found: [C:\Elastic\Agent\data\elastic-agent-9.5.0-SNAPSHOT-56d502\package.version]

pkgversion_common_test.go:46: package version file content: "9.5.0"

The test implementation at testing/integration/ess/pkgversion_common_test.go:53-54 reads the package file and compares it with the binary.version field returned by elastic-agent version --yaml --binary-only. The captured output stops before the assertion diff, so it does not establish whether the binary reported 9.5.0-SNAPSHOT, another value, or whether additional failures dominated the 17-test count. The PR changes shown in the head are focused on osquery/OTel action routing and do not modify this version-resolution code path.

Evidence

• Build: https://buildkite.com/elastic/elastic-agent/builds/42800
• Job/step: windows:amd64:tier2:sudo:default:platform-ingest-elastic-agent-windows-2016-1782867684
• Key log excerpt: === FAIL: testing/integration/ess TestInstallServersWithBasePath/check_agent_package_version; package version file content: "9.5.0"; DONE 172 tests, 125 skipped, 17 failures

Verification

Not run: the captured artifact contains only the final 149 log lines and no assertion diff or complete failure report. No source change was made because the available evidence is insufficient to identify a safe code fix.

Follow-up

The complete TEST-report.html or integration report is required to determine whether the package-version mismatch is the sole actionable failure and to identify the other 16 failures.

---

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.