diff --git a/doc/plans/Innovation-4.3-ipv6-dual-stack.md b/doc/plans/Innovation-4.3-ipv6-dual-stack.md index 9d30dd23..6e557f4c 100644 --- a/doc/plans/Innovation-4.3-ipv6-dual-stack.md +++ b/doc/plans/Innovation-4.3-ipv6-dual-stack.md @@ -1,23 +1,23 @@ ## Sections -1. [1. Overview](#overview) -2. [2. Today](#today) -3. [3. Design](#design) -4. [4. eBPF v6 programs](#ebpf) -5. [5. Listener](#listener) -6. [6. Canonical Destination v6](#canon) -7. [7. Integration](#integration) -8. [8. Tests](#tests) -9. [9. Risks](#risks) -10. [10. Milestones](#milestones) +1. [1. Overview](#1-overview--goals) +2. [2. Implementation status](#2-implementation-status) +3. [3. Address-family semantics](#3-address-family-semantics) +4. [4. eBPF programs](#4-ebpf-programs) +5. [5. Shared audit ABI and telemetry](#5-shared-audit-abi-and-telemetry) +6. [6. Listener and forwarding](#6-listener-and-forwarding) +7. [7. Tests](#7-tests) +8. [8. Remaining native IPv6 work](#8-remaining-native-ipv6-work) +9. [9. Risks](#9-risks) +10. [10. Milestones](#10-milestones) **GPA** · **Direction 4.3** · **Network** # Detailed Design — IPv6 / Dual-stack Support -Extend the redirect, listener, canonical model, and rule engine to handle IPv6 fabric endpoints uniformly with IPv4. Closes the gap on dual-stack VMs. +Add dual-stack protection in two stages. The implemented first stage closes the IPv4-mapped IPv6 bypass (`::ffff:a.b.c.d`) while preserving the existing IPv4 policy and forwarding model. Native IPv6 fabric endpoints remain a later stage. -**Files affected:** `linux-ebpf/sk_lookup.bpf.c`, `ebpf/redirect.bpf.c`, `proxy_agent/src/proxy/proxy_server.rs`, `proxy_agent/src/proxy/canonical/destination.rs`. +**Primary files affected:** `linux-ebpf/ebpf_cgroup.c`, `ebpf/redirect.bpf.c`, `shared-ebpf/include/gpa_audit_event.h`, `proxy_agent/src/redirector/`, and `proxy_agent/src/proxy/`. > **Prerequisites:** [4.2 Core eBPF unification](Innovation-4.2-core-unify-ebpf.md) @@ -27,84 +27,129 @@ Extend the redirect, listener, canonical model, and rule engine to handle IPv6 f |----------------------------|------------|---------|------------------| | **Medium** future-proofing | **Medium** | **Low** | **eBPF + agent** | -### Goals +### Current goals -- IPv6 fabric link-local addresses (e.g. `fe80::a9fe:a9fe`) caught by eBPF and routed through agent. -- Canonical destination enum unified across families; rule engine sees one `Destination::Imds` regardless of family. -- Defeat IPv4-mapped IPv6 bypasses (pentest C7) at the kernel layer. +- Intercept IPv4-mapped IPv6 destinations such as `::ffff:169.254.169.254`. +- Reuse the existing IPv4 policy-map keys and original-destination forwarding path. +- Attach both connect4 and connect6 programs on Linux and Windows. +- Emit the connect address family in request and aggregate telemetry. +- Keep existing IPv4 behavior and released Windows audit-layout compatibility. -## 2. Today +### Deferred goals -Redirect handles IPv4 only. Dual-stack VMs that route fabric over v6 (uncommon today but increasing) bypass the agent. The canonical model in direction 2.1 already plans for v6 typed destinations; this direction wires it through the kernel. +- Redirect native IPv6 fabric endpoints, including link-local addresses. +- Store full 128-bit original destinations in the policy and audit models. +- Forward requests to native IPv6 upstream endpoints. +- Bind and expose a native IPv6 proxy listener where required. -## 3. Design +## 2. Implementation status -- Add v6 sibling programs: `cgroup_connect6`, `sk_lookup_v6`. -- Listener binds IPv6 socket with `IPV6_V6ONLY=0` dual-stack on Linux, or two sockets where dual-stack is unavailable. -- Canonical `Destination` resolves IPv4-mapped IPv6 (`::ffff:a.b.c.d`) to the v4 destination — there is exactly one `Destination::Imds` regardless of family. -- Per-destination address tables published to BPF programs via a `BPF_MAP_TYPE_HASH` keyed on a 16-byte normalized address. +The current implementation supports IPv4-mapped IPv6 only: -## 4. eBPF v6 Programs +- Linux attaches `connect4` and `connect6` cgroup programs. +- Windows loads `authorize_connect4` and `authorize_connect6` and retains both eBPF links for the object lifetime. +- The connect6 programs recognize the `::ffff:0:0/96` prefix, extract the low-order IPv4 address, and look it up in the existing IPv4 policy map. +- A matched connection is redirected to IPv4-mapped loopback while remaining usable by the originating dual-stack socket. +- Native IPv6 addresses do not match the IPv4 policy map and pass through unchanged. +- No new native IPv6 endpoint configuration is introduced in this stage. - SEC("cgroup/connect6") - int gpa_connect6(struct bpf_sock_addr *ctx) { - struct in6_addr dst; - __builtin_memcpy(&dst, ctx->user_ip6, sizeof(dst)); - if (!is_fabric_dest6(&dst, bpf_ntohs(ctx->user_port))) return 1; - // Redirect: rewrite to agent's v6 listener - set_user_dest_v6(ctx, &agent_v6, agent_port); - return 1; - } +## 3. Address-family semantics -- `is_fabric_dest6` recognizes the v6 link-local equivalent (typically `fe80::a9fe:a9fe` if used) and IPv4-mapped forms. -- SO_ORIGINAL_DST equivalent for v6 via `IP6T_SO_ORIGINAL_DST`; reachable from user space. +The telemetry field describes the connect hook/API family observed by eBPF, not necessarily the packet transport used after mapped-address conversion. -## 5. Listener +| Platform | Application destination | Observed hook | Telemetry | +|----------|-------------------------|---------------|-----------| +| Linux | IPv4 `169.254.169.254` | `connect4` | `IPv4` | +| Linux | Mapped `::ffff:169.254.169.254` passed as `sockaddr_in6` | `connect6` | `IPv6` | +| Windows (current eBPF-for-Windows behavior) | IPv4 `169.254.169.254` | `authorize_connect4` | `IPv4` | +| Windows (current eBPF-for-Windows behavior) | Mapped `::ffff:169.254.169.254` on a dual-mode socket | normalized to `authorize_connect4` | `IPv4` | +| Either platform | Native IPv6 | connect6 path, no IPv4 policy match | not redirected | -- Bind `[::1]:3080` in addition to `127.0.0.1:3080` (or single dual-stack socket). -- Original destination read on accept via family-appropriate `getsockopt`. -- Listener exposed via attestation endpoint (3.3) with all bound addresses. +Linux hook selection follows the address family supplied to `connect()`. Windows currently classifies IPv4-mapped dual-stack connections as IPv4 before the GPA hook. The Windows connect6 mapped-address logic remains as a compatibility fallback if eBPF-for-Windows later aligns with Linux behavior. See [eBPF-for-Windows issue #5536](https://github.com/microsoft/ebpf-for-windows/issues/5536). -## 6. Canonical Destination v6 +## 4. eBPF programs - impl Destination { - pub fn from_ip(ip: IpAddr, port: u16) -> Destination { - let v4 = match ip { - IpAddr::V4(v) => Some(v), - IpAddr::V6(v) => v.to_ipv4_mapped(), - }; - match (v4, port) { - (Some(Ipv4Addr::new(169,254,169,254)), 80) => Destination::Imds, - (Some(Ipv4Addr::new(168,63,129,16)), 80) => Destination::WireServer, - (Some(Ipv4Addr::new(168,63,129,16)), 32526) => Destination::HostGaPlugin, - _ => Destination::Unknown { /* ... */ }, - } - } - } +The Linux and Windows connect6 programs use the same decision model: -## 7. Integration +```c +if (!get_ipv4_mapped_address(ctx, &destination_ipv4)) + return PROCEED; -- Loader (4.2) detects v6 enablement on the host and loads v6 programs only when needed. -- PoP token (1.1) `dip` claim is always a 16-byte normalized form so signatures cover both families. -- Telemetry: per-family labels on `gpa_requests_total`. +key = { destination_ipv4, destination_port, protocol }; +policy = bpf_map_lookup_elem(&policy_map, &key); +if (policy) { + record_original_destination_and_family(); + redirect_to_ipv4_mapped_loopback(policy); +} +``` -## 8. Tests +Linux carries the original destination and family through `local_map` until the TCP source port is available. Windows writes the same information to `audit_map` or the WFP redirect context. -- Dual-stack pod test: v4 and v6 requests both reach the agent and produce identical `Destination`. -- Pentest C7 v6 variants: all map to `Destination::Imds` after canonicalization. -- Linkup test for hosts without v6 — programs not loaded; no warnings. +## 5. Shared audit ABI and telemetry + +`gpa_audit_event` is shared by Linux and Windows and now contains: + +- Existing identity, process, original IPv4 destination, and port fields. +- `address_family`, normalized to `GPA_ADDRESS_FAMILY_IPV4` (`4`) or `GPA_ADDRESS_FAMILY_IPV6` (`6`). +- A reserved word for future ABI-compatible metadata. + +The canonical audit value is 28 bytes (`[u32; 7]`). The Rust decoder also accepts the officially released 24-byte legacy Windows layout and treats it as IPv4. The unreleased 20-byte intermediate layout is intentionally not supported. + +The family is propagated through `AuditEntry` and `TcpConnectionContext` into: + +- Per-request `ProxySummary` JSON as `addressFamily: "IPv4" | "IPv6"`. +- `ProxyConnectionSummary` aggregate status. +- The aggregation key, so IPv4 and IPv6 requests do not collapse into one bucket. + +Older serialized summaries that omit `addressFamily` default to `IPv4`. + +## 6. Listener and forwarding + +The current mapped-IPv6 stage keeps the existing IPv4 proxy listener and IPv4 upstream forwarding path. The audit record stores the extracted IPv4 destination, so authorization and forwarding remain unchanged. + +A native IPv6 listener, 128-bit original-destination storage, and native IPv6 upstream sender are not part of the current implementation. + +## 7. Tests + +Implemented validation includes: + +- Shared audit-layout round trips and IPv6 family decoding. +- Released legacy Windows audit-layout decoding as IPv4. +- Family-aware connection-summary aggregation and JSON serialization. +- Windows-target Cargo checks and focused Rust tests. +- Windows eBPF compilation with both `cgroup/connect4` and `cgroup/connect6` sections. + +Required environment tests: + +- Linux dual-mode client using `sockaddr_in6(::ffff:a.b.c.d)` reports `IPv6` and is redirected. +- Windows dual-mode client may report `IPv4` because the platform normalizes the mapped address before the hook; it must still be redirected. +- Native IPv6 destinations pass through unchanged until the next milestone. + +## 8. Remaining native IPv6 work + +1. Define production native IPv6 fabric endpoint addresses and configuration. +2. Replace the IPv4-only policy key with a normalized 16-byte address plus family, port, and protocol. +3. Expand audit and connection models to retain a full 128-bit original destination. +4. Add native IPv6 listener and upstream forwarding support. +5. Fold mapped and native representations into canonical endpoint identities. +6. Add native IPv6 end-to-end and bypass tests. ## 9. Risks -- **Fabric v6 endpoints not finalized** in some regions — make destinations data-driven via the BPF map so production can update without redeploying eBPF. -- **Dual-stack socket semantics** vary on Windows — keep two sockets there. +- **Telemetry interpretation:** `addressFamily` records the observed connect family, not guaranteed on-wire IP transport. Platform normalization makes mapped-address results differ between Linux and Windows. +- **Kernel variation:** Linux cgroup connect hooks require kernel 4.17 or later; enterprise distributions may backport them. IPv6 or mapped-address support can also be disabled by host configuration. +- **Windows behavior may change:** the connect6 fallback must remain tested even though current eBPF-for-Windows routes mapped connections through connect4. +- **ABI coordination:** shared C map layouts and Rust `[u32; N]` representations must change together. +- **Native endpoint uncertainty:** fabric IPv6 addresses are not finalized in all environments. ## 10. Milestones -| M | Deliverable | Exit | -|-----|------------------------------|---------------------------------------------| -| M1 | v6 listener + canonical fold | v4 behavior unchanged | -| M2 | connect6 + sk_lookup_v6 | v6 fabric traffic captured in dual-stack VM | -| M3 | Data-driven dest table | Region rollout without rebuild | +| M | Deliverable | Status / exit criteria | +|---|-------------|------------------------| +| M1 | Mapped-IPv6 interception on Linux | Implemented: connect6 maps `::ffff:a.b.c.d` to existing IPv4 policy and forwarding | +| M2 | Mapped-IPv6 compatibility on Windows | Implemented: both links attached; current connect4 normalization and connect6 fallback supported | +| M3 | Cross-platform family telemetry | Implemented: shared audit ABI and `addressFamily` request/aggregate telemetry | +| M4 | Native IPv6 interception and forwarding | Planned: 128-bit policy, audit, listener, sender, and end-to-end tests | +| M5 | Data-driven native endpoint table | Planned: region updates without eBPF redeployment | Detail design for direction 4.3. Parent: [Innovation-Directions.md](Innovation-Directions.md). diff --git a/e2etest/GuestProxyAgentTest/LinuxScripts/IMDSPingTest.sh b/e2etest/GuestProxyAgentTest/LinuxScripts/IMDSPingTest.sh index 3e4d3d3a..3c400846 100755 --- a/e2etest/GuestProxyAgentTest/LinuxScripts/IMDSPingTest.sh +++ b/e2etest/GuestProxyAgentTest/LinuxScripts/IMDSPingTest.sh @@ -3,11 +3,12 @@ # Copyright (c) Microsoft Corporation # SPDX-License-Identifier: MIT echo "$(date -u +"%Y-%m-%dT%H:%M:%SZ") - imdsSecureChannelEnabled=$imdsSecureChannelEnabled" +echo "$(date -u +"%Y-%m-%dT%H:%M:%SZ") - ipv6DualStackSupported=$ipv6DualStackSupported" -# make 10 requests if any failed, will failed the test for tcp port scalability config +# make 10 requests if any failed, will failed the test for i in {1..10}; do url="http://169.254.169.254/metadata/instance?api-version=2020-06-01" - statusCode=$(curl -s -o /dev/null -w "%{http_code}" -H "Metadata:True" $url) + statusCode=$(curl --noproxy "*" -s -o /dev/null -w "%{http_code}" -H "Metadata:True" "$url") if [ $statusCode -eq 200 ]; then echo "$(date -u +"%Y-%m-%dT%H:%M:%SZ") - Response status code is OK (200)" else @@ -16,7 +17,7 @@ for i in {1..10}; do fi sleep 1 - authorizationHeader=$(curl -s -I -H "Metadata:True" $url | grep -Fi "x-ms-azure-host-authorization") + authorizationHeader=$(curl --noproxy "*" -s -I -H "Metadata:True" "$url" | grep -Fi "x-ms-azure-host-authorization") if [ "${imdsSecureChannelEnabled,,}" = "true" ] # case insensitive comparison then if [ "$authorizationHeader" = "" ]; then @@ -37,4 +38,62 @@ for i in {1..10}; do fi done +if [ "${ipv6DualStackSupported,,}" != "true" ]; then + echo "$(date -u +"%Y-%m-%dT%H:%M:%SZ") - IPv6 dual stack is not supported on this GPA. Skipping the IPv4-mapped IPv6 ping test." + exit 0 +fi + +if [ ! -e /proc/net/if_inet6 ] || + { [ -r /proc/sys/net/ipv6/conf/all/disable_ipv6 ] && [ "$(cat /proc/sys/net/ipv6/conf/all/disable_ipv6)" -eq 1 ]; }; then + echo "$(date -u +"%Y-%m-%dT%H:%M:%SZ") - IPv6 is not supported or is disabled on this VM. Skipping the IPv4-mapped IPv6 ping test." + exit 0 +fi + +if ! curl --version | grep -qE '^Features:.*[[:space:]]IPv6([[:space:]]|$)'; then + echo "$(date -u +"%Y-%m-%dT%H:%M:%SZ") - curl does not support IPv6. Skipping the IPv4-mapped IPv6 ping test." + exit 0 +fi + +# make 10 requests if any failed, will failed the test +for i in {1..10}; do + ipv6_dual_stack_url="http://[::ffff:169.254.169.254]/metadata/instance?api-version=2020-06-01" + statusCode=$(curl --noproxy "*" --ipv6 --silent --show-error --output /dev/null --write-out "%{http_code}" -H "Metadata:True" -H "Host: 169.254.169.254" "$ipv6_dual_stack_url") + curlExitCode=$? + if [ $curlExitCode -ne 0 ]; then + echo "$(date -u +"%Y-%m-%dT%H:%M:%SZ") - IPv6 Dual Stack Ping test failed. curl exit code is $curlExitCode" + exit -1 + elif [ "$statusCode" -eq 200 ]; then + echo "$(date -u +"%Y-%m-%dT%H:%M:%SZ") - IPv6 Dual Stack Response status code is OK (200)" + else + echo "$(date -u +"%Y-%m-%dT%H:%M:%SZ") - IPv6 Dual Stack Ping test failed. Response status code is $statusCode" + exit -1 + fi + sleep 1 + + responseHeaders=$(curl --noproxy "*" --ipv6 --silent --show-error --head -H "Metadata:True" -H "Host: 169.254.169.254" "$ipv6_dual_stack_url") + curlExitCode=$? + if [ $curlExitCode -ne 0 ]; then + echo "$(date -u +"%Y-%m-%dT%H:%M:%SZ") - IPv6 Dual Stack Ping test failed while reading response headers. curl exit code is $curlExitCode" + exit -1 + fi + authorizationHeader=$(printf '%s\n' "$responseHeaders" | grep -Fi "x-ms-azure-host-authorization") + if [ "${imdsSecureChannelEnabled,,}" = "true" ] # case insensitive comparison + then + if [ "$authorizationHeader" = "" ]; then + echo "$(date -u +"%Y-%m-%dT%H:%M:%SZ") - IPv6 Dual Stack Response authorization header not exist" + exit -1 + else + echo "$(date -u +"%Y-%m-%dT%H:%M:%SZ") - IPv6 Dual Stack Response authorization header exists as expected" + fi + sleep 1 + else + if [ "$authorizationHeader" = "" ]; then + echo "$(date -u +"%Y-%m-%dT%H:%M:%SZ") - IPv6 Dual Stack Response authorization header not exist as expected" + else + echo "$(date -u +"%Y-%m-%dT%H:%M:%SZ") - IPv6 Dual Stack Response authorization header exists" + exit -1 + fi + sleep 1 + fi +done exit 0 \ No newline at end of file diff --git a/e2etest/GuestProxyAgentTest/Scripts/IMDSPingTest.ps1 b/e2etest/GuestProxyAgentTest/Scripts/IMDSPingTest.ps1 index 4d9dcae5..dc66b2b3 100644 --- a/e2etest/GuestProxyAgentTest/Scripts/IMDSPingTest.ps1 +++ b/e2etest/GuestProxyAgentTest/Scripts/IMDSPingTest.ps1 @@ -7,8 +7,35 @@ param ( ) Write-Output "$((Get-Date).ToUniversalTime()) - imdsSecureChannelEnabled=$imdsSecureChannelEnabled" +function Test-IsIpv6UnsupportedError { + param ( + [System.Exception]$Exception + ) + + while ($null -ne $Exception) { + if ($Exception -is [System.PlatformNotSupportedException] -or + $Exception -is [System.NotSupportedException]) { + return $true + } + + if ($Exception -is [System.Net.Sockets.SocketException]) { + return $Exception.SocketErrorCode -in @( + [System.Net.Sockets.SocketError]::AddressFamilyNotSupported, + [System.Net.Sockets.SocketError]::AddressNotAvailable, + [System.Net.Sockets.SocketError]::NetworkUnreachable, + [System.Net.Sockets.SocketError]::OperationNotSupported, + [System.Net.Sockets.SocketError]::ProtocolNotSupported + ) + } + + $Exception = $Exception.InnerException + } + + return $false +} + $i = 0 -# make 10 requests if any failed, will failed the test for tcp port scalability config +# make 10 requests if any failed, will failed the test while ($i -lt 10) { try { $url = "http://169.254.169.254/metadata/instance?api-version=2020-06-01" @@ -52,4 +79,82 @@ while ($i -lt 10) { start-sleep -Seconds 1 $i++ } + +if (-not [System.Net.Sockets.Socket]::OSSupportsIPv6) { + Write-Warning "$((Get-Date).ToUniversalTime()) - IPv6 is not supported on this VM. Skipping the IPv4-mapped IPv6 ping test." + exit 0 +} + +$i = 0 +while ($i -lt 10) { + $tcpClient = $null + $reader = $null + try { + $tcpClient = [System.Net.Sockets.TcpClient]::new([System.Net.Sockets.AddressFamily]::InterNetworkV6) + $tcpClient.Client.DualMode = $true + $tcpClient.Connect([System.Net.IPAddress]::Parse("::ffff:169.254.169.254"), 80) + + $stream = $tcpClient.GetStream() + $stream.ReadTimeout = 30000 + $stream.WriteTimeout = 30000 + $request = "GET /metadata/instance?api-version=2020-06-01 HTTP/1.1`r`nHost: 169.254.169.254`r`nMetadata: True`r`nConnection: close`r`n`r`n" + $requestBytes = [System.Text.Encoding]::ASCII.GetBytes($request) + $stream.Write($requestBytes, 0, $requestBytes.Length) + + $reader = [System.IO.StreamReader]::new($stream, [System.Text.Encoding]::ASCII) + $statusLine = $reader.ReadLine() + if ($statusLine -match '^HTTP/\d(?:\.\d)? 200(?:\s|$)') { + Write-Output "$((Get-Date).ToUniversalTime()) - IPv4-mapped IPv6 ping test response status code is OK (200)" + } + else { + Write-Error "$((Get-Date).ToUniversalTime()) - IPv4-mapped IPv6 ping test failed. Response status is '$statusLine'" + exit -1 + } + + $responseHeaders = @{} + while (($headerLine = $reader.ReadLine()) -ne $null -and $headerLine.Length -gt 0) { + $separator = $headerLine.IndexOf(':') + if ($separator -gt 0) { + $responseHeaders[$headerLine.Substring(0, $separator).Trim()] = $headerLine.Substring($separator + 1).Trim() + } + } + + if ("$imdsSecureChannelEnabled" -ieq "true") { # case insensitive comparison + if ($null -eq $responseHeaders["x-ms-azure-host-authorization"]) { + Write-Error "$((Get-Date).ToUniversalTime()) - IPv4-mapped IPv6 ping test failed. Response does not contain x-ms-azure-host-authorization header" + exit -1 + } + else { + Write-Output "$((Get-Date).ToUniversalTime()) - IPv4-mapped IPv6 ping test passed. Response contains x-ms-azure-host-authorization header" + } + } + else { + if ($null -eq $responseHeaders["x-ms-azure-host-authorization"]) { + Write-Output "$((Get-Date).ToUniversalTime()) - IPv4-mapped IPv6 ping test passed. Response does not contain x-ms-azure-host-authorization header as expected" + } + else { + Write-Error "$((Get-Date).ToUniversalTime()) - IPv4-mapped IPv6 ping test failed. Response contains x-ms-azure-host-authorization header" + exit -1 + } + } + } + catch { + if (Test-IsIpv6UnsupportedError -Exception $_.Exception) { + Write-Warning "$((Get-Date).ToUniversalTime()) - IPv6 or dual-stack sockets are not supported on this VM. Skipping the IPv4-mapped IPv6 ping test. Error: $_" + break + } + Write-Error "$((Get-Date).ToUniversalTime()) - IPv4-mapped IPv6 request failed: $_" + exit -1 + } + finally { + if ($null -ne $reader) { + $reader.Dispose() + } + if ($null -ne $tcpClient) { + $tcpClient.Dispose() + } + } + start-sleep -Seconds 1 + $i++ +} exit 0 \ No newline at end of file diff --git a/e2etest/GuestProxyAgentTest/TestCases/IMDSPingTestCase.cs b/e2etest/GuestProxyAgentTest/TestCases/IMDSPingTestCase.cs index 8e18ec87..18c4ecfa 100644 --- a/e2etest/GuestProxyAgentTest/TestCases/IMDSPingTestCase.cs +++ b/e2etest/GuestProxyAgentTest/TestCases/IMDSPingTestCase.cs @@ -8,17 +8,20 @@ namespace GuestProxyAgentTest.TestCases { public class IMDSPingTestCase : TestCaseBase { - public IMDSPingTestCase(string testCaseName, bool imdsSecureChannelEnabled) : base(testCaseName) + public IMDSPingTestCase(string testCaseName, bool imdsSecureChannelEnabled, bool ipv6DualStackSupported = true) : base(testCaseName) { ImdsSecureChannelEnabled = imdsSecureChannelEnabled; + Ipv6DualStackSupported = ipv6DualStackSupported; } private bool ImdsSecureChannelEnabled { get; set; } + private bool Ipv6DualStackSupported { get; set; } public override async Task StartAsync(TestCaseExecutionContext context) { List<(string, string)> parameterList = new List<(string, string)>(); parameterList.Add(("imdsSecureChannelEnabled", ImdsSecureChannelEnabled.ToString())); + parameterList.Add(("ipv6DualStackSupported", Ipv6DualStackSupported.ToString())); context.TestResultDetails = (await RunScriptViaRunCommandV2Async(context, Constants.IMDS_PING_TEST_SCRIPT_NAME, parameterList, false)).ToTestResultDetails(context.Logger); } } diff --git a/e2etest/GuestProxyAgentTest/TestScenarios/BakedInScenario.cs b/e2etest/GuestProxyAgentTest/TestScenarios/BakedInScenario.cs index f763037a..954f9a09 100644 --- a/e2etest/GuestProxyAgentTest/TestScenarios/BakedInScenario.cs +++ b/e2etest/GuestProxyAgentTest/TestScenarios/BakedInScenario.cs @@ -16,19 +16,21 @@ public override void TestScenarioSetup() } var secureChannelEnabled = false; + // currently IPv6 dual stack is not supported on this bakedIn scenario, + var ipv6DualStackSupported = false; EnableProxyAgentForNewVM = false; AddTestCase(new GuestProxyAgentValidationCase("GuestProxyAgentValidationWithoutMSP", "disabled")); - AddTestCase(new IMDSPingTestCase("IMDSPingTestBeforeEnableMSP", secureChannelEnabled)); + AddTestCase(new IMDSPingTestCase("IMDSPingTestBeforeEnableMSP", secureChannelEnabled, ipv6DualStackSupported)); // enable secure channel after validation to test IMDS connectivity with secure channel enabled, AddTestCase(new EnableProxyAgentCase()); secureChannelEnabled = true; AddTestCase(new GuestProxyAgentValidationCase("GuestProxyAgentValidationWithSecureChannelEnabled", "WireServer Enforce - IMDS Enforce - HostGA Enforce")); - AddTestCase(new IMDSPingTestCase("IMDSPingTestBeforeReboot", secureChannelEnabled)); + AddTestCase(new IMDSPingTestCase("IMDSPingTestBeforeReboot", secureChannelEnabled, ipv6DualStackSupported)); // then reboot to verify the secure channel state is preserved across reboots AddTestCase(new RebootVMCase("RebootVMCaseAfterEnableMSP")); - AddTestCase(new IMDSPingTestCase("IMDSPingTestAfterReboot", secureChannelEnabled)); + AddTestCase(new IMDSPingTestCase("IMDSPingTestAfterReboot", secureChannelEnabled, ipv6DualStackSupported)); } } } diff --git a/ebpf/redirect.bpf.c b/ebpf/redirect.bpf.c index 1acee7c7..5648a741 100644 --- a/ebpf/redirect.bpf.c +++ b/ebpf/redirect.bpf.c @@ -2,6 +2,7 @@ // SPDX-License-Identifier: MIT #include "bpf_helpers.h" +#include "bpf_endian.h" #include "socket.h" // SEC("maps") @@ -47,7 +48,7 @@ check_skip_process_map_entry(uint32_t pid) return 1 if pid found in the skip_process_map. */ inline __attribute__((always_inline)) int -update_audit_map_entry(bpf_sock_addr_t *ctx) +update_audit_map_entry(bpf_sock_addr_t *ctx, uint32_t destination_ipv4, uint32_t address_family) { uint64_t pid_tip = bpf_get_current_pid_tgid(); uint32_t pid = (uint32_t)(pid_tip >> 32); @@ -74,8 +75,9 @@ update_audit_map_entry(bpf_sock_addr_t *ctx) { entry.is_root = (is_admin > 0) ? 1 : 0; } - entry.destination_ipv4 = ctx->user_ip4; // we only support ipv4 so far. + entry.destination_ipv4 = destination_ipv4; entry.destination_port = ctx->user_port; + entry.address_family = address_family; uint16_t source_port = ctx->msg_src_port; if (source_port == 0) { @@ -122,7 +124,7 @@ authorize_v4(bpf_sock_addr_t *ctx) bpf_printk("Found v4 proxy entry value: %u, %u", policy->destination_ip.ipv4, policy->destination_port); // update to the audit map before changing the destination ip and port. - if (update_audit_map_entry(ctx) == 1) + if (update_audit_map_entry(ctx, ctx->user_ip4, GPA_ADDRESS_FAMILY_IPV4) == 1) { bpf_printk("Found skip process entry, skip the redirection."); return BPF_SOCK_ADDR_VERDICT_PROCEED_SOFT; @@ -153,3 +155,59 @@ int authorize_connect4(bpf_sock_addr_t *ctx) { return authorize_v4(ctx); } + +inline __attribute__((always_inline)) int +get_ipv4_mapped_address(bpf_sock_addr_t *ctx, uint32_t *destination_ipv4) +{ + if (ctx->user_ip6[0] != 0 || + ctx->user_ip6[1] != 0 || + ctx->user_ip6[2] != bpf_htonl(0x0000ffff)) + { + return 0; + } + + *destination_ipv4 = ctx->user_ip6[3]; + return 1; +} + +// SEC("cgroup/connect6") +#pragma clang section text = "cgroup/connect6" +int authorize_connect6(bpf_sock_addr_t *ctx) +{ + // Check if the destination address is an IPv4-mapped IPv6 address. + // While the current eBPF_for_Windows detects the IPv4-mapped address, + // explicitly classify/convert dual-stack IPv4-mapped connections as IPv4. + // refer to https://github.com/microsoft/ebpf-for-windows/issues/5536 + // We keep this logic here to support dual-stack IPv4-mapped connections in connect6, + // just in case windows eBPF may change the behavior to align with Linux eBPF. + uint32_t destination_ipv4; + if (get_ipv4_mapped_address(ctx, &destination_ipv4) == 0) + { + // Native IPv6 destinations are not redirected by the IPv4 policy map. + return BPF_SOCK_ADDR_VERDICT_PROCEED_SOFT; + } + + destination_entry_t entry = {0}; + entry.destination_ip.ipv4 = destination_ipv4; + entry.destination_port = ctx->user_port; + entry.protocol = ctx->protocol; + + destination_entry_t *policy = bpf_map_lookup_elem(&policy_map, &entry); + if (policy != NULL) + { + bpf_printk("Found IPv4-mapped proxy entry."); + if (update_audit_map_entry(ctx, destination_ipv4, GPA_ADDRESS_FAMILY_IPV6) == 1) + { + bpf_printk("Found skip process entry, skip the redirection."); + return BPF_SOCK_ADDR_VERDICT_PROCEED_SOFT; + } + + ctx->user_ip6[0] = 0; + ctx->user_ip6[1] = 0; + ctx->user_ip6[2] = bpf_htonl(0x0000ffff); + ctx->user_ip6[3] = policy->destination_ip.ipv4; + ctx->user_port = policy->destination_port; + } + + return BPF_SOCK_ADDR_VERDICT_PROCEED_SOFT; +} diff --git a/ebpf/socket.h b/ebpf/socket.h index 693f787c..db8d86ec 100644 --- a/ebpf/socket.h +++ b/ebpf/socket.h @@ -21,4 +21,4 @@ typedef struct gpa_destination_entry destination_entry_t; typedef struct gpa_audit_key sock_addr_audit_key_t; typedef struct gpa_audit_event sock_addr_audit_entry_t; -typedef struct gpa_skip_process_entry sock_addr_skip_process_entry; \ No newline at end of file +typedef struct gpa_skip_process_entry sock_addr_skip_process_entry; diff --git a/linux-ebpf/ebpf_cgroup.c b/linux-ebpf/ebpf_cgroup.c index a58f82f7..ba425899 100644 --- a/linux-ebpf/ebpf_cgroup.c +++ b/linux-ebpf/ebpf_cgroup.c @@ -9,39 +9,43 @@ #include #include #include +#include #include "socket.h" // BPF maps for policy and audit -struct { +struct +{ __uint(type, BPF_MAP_TYPE_HASH); __type(key, struct gpa_skip_process_entry); __type(value, struct gpa_skip_process_entry); __uint(max_entries, 10); } skip_process_map SEC(".maps"); -struct { +struct +{ __uint(type, BPF_MAP_TYPE_HASH); __type(key, struct gpa_destination_entry); __type(value, struct gpa_destination_entry); __uint(max_entries, 10); } policy_map SEC(".maps"); -struct { +struct +{ __uint(type, BPF_MAP_TYPE_LRU_HASH); - __type(key, struct gpa_audit_key); // source port and protocol - __type(value, struct gpa_audit_event); // audit event (canonical struct) - __uint(max_entries, 200); // LRU evicts oldest on overflow + __type(key, struct gpa_audit_key); // source port and protocol + __type(value, struct gpa_audit_event); // audit event (canonical struct) + __uint(max_entries, 200); // LRU evicts oldest on overflow } audit_map SEC(".maps"); -struct { +struct +{ __uint(type, BPF_MAP_TYPE_LRU_HASH); - __type(key, __u64); // pid-tgid or socket cookie + __type(key, __u64); // pid-tgid or socket cookie __type(value, struct gpa_sock_addr_local_entry); __uint(max_entries, 200); } local_map SEC(".maps"); - /* check the current pid in the skip_process map. return 1 if found, otherwise return 0. @@ -63,7 +67,7 @@ check_skip_process_map_entry(__u32 pid) return 1 if pid found in the skip_process_map. */ static __always_inline int -update_local_map_entry(struct bpf_sock_addr *ctx) +update_local_map_entry(struct bpf_sock_addr *ctx, __be32 destination_ipv4, __u32 address_family) { __u64 pid_tip = bpf_get_current_pid_tgid(); __u32 pid = (__u32)(pid_tip >> 32); @@ -77,10 +81,11 @@ update_local_map_entry(struct bpf_sock_addr *ctx) entry.process_id = pid; __u32 uid = (__u32)(bpf_get_current_uid_gid() >> 32); entry.logon_id = uid; - entry.is_root = (uid == 0) ? 1 : 0; // root uid is 0. - entry.destination_ipv4 = ctx->user_ip4; // we only support ipv4 so far. + entry.is_root = (uid == 0) ? 1 : 0; // root uid is 0. + entry.destination_ipv4 = destination_ipv4; entry.destination_port = ctx->user_port; entry.protocol = ctx->protocol; + entry.address_family = address_family; __u64 ret = bpf_map_update_elem(&local_map, &pid_tip, &entry, 0); if (ret != 0) @@ -110,7 +115,7 @@ authorize_v4(struct bpf_sock_addr *ctx) bpf_printk("authorize_v4: Found v4 proxy entry value: %u, %u", policy->destination_ip.ipv4, policy->destination_port); // update to the audit map before changing the destination ip and port. - if (update_local_map_entry(ctx) == 1) + if (update_local_map_entry(ctx, ctx->user_ip4, GPA_ADDRESS_FAMILY_IPV4) == 1) { bpf_printk("authorize_v4: Found skip process entry, skip the redirection."); return BPF_SOCK_ADDR_VERDICT_PROCEED; @@ -139,10 +144,63 @@ authorize_v4(struct bpf_sock_addr *ctx) SEC("cgroup/connect4") int connect4(struct bpf_sock_addr *ctx) { - __u64 cookie = bpf_get_socket_cookie(ctx); return authorize_v4(ctx); } +/// @brief Extract the IPv4 address from an IPv4-mapped IPv6 address. +/// @param ctx The socket address context containing the IPv6 address. +/// @param destination_ipv4 Pointer to store the extracted IPv4 address. +/// @return 1 if the address is IPv4-mapped, 0 otherwise. +static __always_inline int +get_ipv4_mapped_address(struct bpf_sock_addr *ctx, __be32 *destination_ipv4) +{ + if (ctx->user_ip6[0] != 0 || + ctx->user_ip6[1] != 0 || + ctx->user_ip6[2] != bpf_htonl(0x0000ffff)) + { + return 0; + } + + *destination_ipv4 = ctx->user_ip6[3]; + return 1; +} + +SEC("cgroup/connect6") +int connect6(struct bpf_sock_addr *ctx) +{ + __be32 destination_ipv4; + if (get_ipv4_mapped_address(ctx, &destination_ipv4) == 0) + { + // Native IPv6 destinations are not supported yet and must remain unchanged. + return BPF_SOCK_ADDR_VERDICT_PROCEED; + } + + struct gpa_destination_entry entry = {0}; + entry.destination_ip.ipv4 = destination_ipv4; + entry.destination_port = ctx->user_port; + entry.protocol = ctx->protocol; + + struct gpa_destination_entry *policy = bpf_map_lookup_elem(&policy_map, &entry); + if (policy != NULL) + { + bpf_printk("connect6: Found IPv4-mapped proxy entry."); + if (update_local_map_entry(ctx, destination_ipv4, GPA_ADDRESS_FAMILY_IPV6) == 1) + { + bpf_printk("connect6: Found skip process entry, skip the redirection."); + return BPF_SOCK_ADDR_VERDICT_PROCEED; + } + + // Keep the socket in AF_INET6 and redirect it to IPv4-mapped loopback. + ctx->user_ip6[0] = 0; + ctx->user_ip6[1] = 0; + ctx->user_ip6[2] = bpf_htonl(0x0000ffff); + ctx->user_ip6[3] = policy->destination_ip.ipv4; + ctx->user_port = policy->destination_port; + } + + return BPF_SOCK_ADDR_VERDICT_PROCEED; +} + static __always_inline int update_audit_map_entry_sk(__u32 local_port, struct gpa_sock_addr_local_entry *local_entry) { @@ -156,6 +214,7 @@ update_audit_map_entry_sk(__u32 local_port, struct gpa_sock_addr_local_entry *lo entry.is_root = local_entry->is_root; entry.destination_ipv4 = local_entry->destination_ipv4; entry.destination_port = local_entry->destination_port; + entry.address_family = local_entry->address_family; __u64 ret = bpf_map_update_elem(&audit_map, &key, &entry, 0); if (ret != 0) @@ -171,7 +230,7 @@ update_audit_map_entry_sk(__u32 local_port, struct gpa_sock_addr_local_entry *lo } static __always_inline int -trace_v4(struct pt_regs *ctx, struct sock *sk) +trace_tcp_connect(struct sock *sk) { // CO-RE relocatable reads of kernel struct sock fields. // BPF_CORE_READ relocates each field offset on the KERNEL-side type @@ -180,21 +239,13 @@ trace_v4(struct pt_regs *ctx, struct sock *sk) // local scalars (no preserve_access_index), so their offsets are NOT // relocated - this is required, otherwise the verifier rejects writes that // would land outside our local stack copy. - __u16 skc_family = BPF_CORE_READ(sk, __sk_common.skc_family); - if (skc_family != AF_INET) - { - // Only support IPv4. - return 0; - } - __be32 skc_daddr = BPF_CORE_READ(sk, __sk_common.skc_daddr); - __be16 skc_dport = BPF_CORE_READ(sk, __sk_common.skc_dport); __u16 skc_num = BPF_CORE_READ(sk, __sk_common.skc_num); __u64 pid_tgid = bpf_get_current_pid_tgid(); __u32 pid = (__u32)(pid_tgid >> 32); if (check_skip_process_map_entry(pid) == 1) { - bpf_printk("trace_v4: Found skip process entry %u, skip the trace.", pid); + bpf_printk("trace_tcp_connect: Found skip process entry %u, skip the trace.", pid); return 0; } @@ -206,53 +257,23 @@ trace_v4(struct pt_regs *ctx, struct sock *sk) __u64 ret = bpf_map_delete_elem(&local_map, &pid_tgid); if (ret != 0) { - bpf_printk("trace_v4: Failed to delete local map entry with results:%u.", ret); + bpf_printk("trace_tcp_connect: Failed to delete local map entry with results:%u.", ret); } else { - bpf_printk("trace_v4: Deleted local map entry with key:%u.", pid_tgid); + bpf_printk("trace_tcp_connect: Deleted local map entry with key:%u.", pid_tgid); } return 0; } - struct gpa_destination_entry entry = {0}; - entry.destination_ip.ipv4 = skc_daddr; - entry.destination_port = skc_dport; - entry.protocol = IPPROTO_TCP; - // Find the entry in the policy map. - struct gpa_destination_entry *policy = bpf_map_lookup_elem(&policy_map, &entry); - if (policy != NULL) - { - __u32 uid = (__u32)(bpf_get_current_uid_gid() >> 32); - struct gpa_audit_key key = {0}; - key.protocol = IPPROTO_TCP; - key.source_port = skc_num; - - struct gpa_audit_event audit_entry = {0}; - audit_entry.process_id = pid; - audit_entry.logon_id = uid; - audit_entry.is_root = (uid == 0) ? 1 : 0; // root uid is 0. - audit_entry.destination_ipv4 = skc_daddr; - audit_entry.destination_port = skc_dport; - - __u64 ret = bpf_map_update_elem(&audit_map, &key, &audit_entry, 0); - if (ret != 0) - { - bpf_printk("trace_v4: Failed to update audit map entry with results:%u.", ret); - } - else - { - bpf_printk("trace_v4: Updated audit map entry with local port:%u.", key.source_port); - } - } - return 0; } -SEC("kprobe/tcp_v4_connect") -int BPF_KPROBE(tcp_v4_connect, struct sock *sk) +SEC("kprobe/tcp_connect") // ELF program type/section metadata +int BPF_KPROBE(tcp_connect_probe, // eBPF program name used by Aya + struct sock *sk) { - return trace_v4(ctx, sk); + return trace_tcp_connect(sk); } char _license[] SEC("license") = "GPL"; \ No newline at end of file diff --git a/proxy_agent/src/proxy/proxy_connection.rs b/proxy_agent/src/proxy/proxy_connection.rs index 20fe40db..f6867b43 100644 --- a/proxy_agent/src/proxy/proxy_connection.rs +++ b/proxy_agent/src/proxy/proxy_connection.rs @@ -7,7 +7,7 @@ use crate::common::config; use crate::common::error::Error; use crate::common::result::Result; use crate::proxy::Claims; -use crate::redirector::{self, AuditEntry}; +use crate::redirector::{self, AddressFamily, AuditEntry}; use crate::shared_state::proxy_server_wrapper::ProxyServerSharedState; use crate::shared_state::redirector_wrapper::RedirectorSharedState; use hyper::body::Bytes; @@ -56,6 +56,7 @@ pub struct TcpConnectionContext { pub claims: Option, pub destination_ip: Option, // currently, we only support IPv4 pub destination_port: u16, + pub address_family: Option, sender: std::result::Result>, String>, logger: ConnectionLogger, } @@ -72,67 +73,71 @@ impl TcpConnectionContext { let client_source_port = client_addr.port(); let mut logger = ConnectionLogger::new(id, 0); - let (claims, destination_ip, destination_port, sender) = match Self::get_audit_entry( - &client_addr, - &redirector_shared_state, - &mut logger, - #[cfg(windows)] - raw_socket_id, - ) - .await - { - Ok(audit_entry) => { - let claims = match Claims::from_audit_entry( - &audit_entry, - client_source_ip, - client_source_port, - proxy_server_shared_state, - ) - .await - { - Ok(claims) => Some(claims), - Err(e) => { - logger.write( - LoggerLevel::Error, - format!("Failed to get claims from audit entry: {e}"), - ); - // return None for claims - None - } - }; - - let host_ip = audit_entry.destination_ipv4_addr().to_string(); - let host_port = audit_entry.destination_port_in_host_byte_order(); - let mut cloned_logger = logger.clone(); - let fun = move |message: String| { - cloned_logger.write(LoggerLevel::Warn, message); - }; - let sender = match hyper_client::build_http_sender(&host_ip, host_port, fun).await { - Ok(sender) => { - logger.write( - LoggerLevel::Trace, - "Successfully created http sender".to_string(), - ); - Ok(Arc::new(Mutex::new(Client { sender }))) - } - Err(e) => Err(e.to_string()), - }; - - ( - claims, - Some(audit_entry.destination_ipv4_addr()), - host_port, - sender, - ) - } - Err(e) => { - logger.write( - LoggerLevel::Warn, - "This tcp connection may send to proxy agent tcp listener directly".to_string(), - ); - (None, None, 0, Err(e.to_string())) - } - }; + let (claims, destination_ip, destination_port, address_family, sender) = + match Self::get_audit_entry( + &client_addr, + &redirector_shared_state, + &mut logger, + #[cfg(windows)] + raw_socket_id, + ) + .await + { + Ok(audit_entry) => { + let claims = match Claims::from_audit_entry( + &audit_entry, + client_source_ip, + client_source_port, + proxy_server_shared_state, + ) + .await + { + Ok(claims) => Some(claims), + Err(e) => { + logger.write( + LoggerLevel::Error, + format!("Failed to get claims from audit entry: {e}"), + ); + // return None for claims + None + } + }; + + let host_ip = audit_entry.destination_ipv4_addr().to_string(); + let host_port = audit_entry.destination_port_in_host_byte_order(); + let mut cloned_logger = logger.clone(); + let fun = move |message: String| { + cloned_logger.write(LoggerLevel::Warn, message); + }; + let sender = + match hyper_client::build_http_sender(&host_ip, host_port, fun).await { + Ok(sender) => { + logger.write( + LoggerLevel::Trace, + "Successfully created http sender".to_string(), + ); + Ok(Arc::new(Mutex::new(Client { sender }))) + } + Err(e) => Err(e.to_string()), + }; + + ( + claims, + Some(audit_entry.destination_ipv4_addr()), + host_port, + Some(audit_entry.address_family), + sender, + ) + } + Err(e) => { + logger.write( + LoggerLevel::Warn, + "This tcp connection may send to proxy agent tcp listener directly" + .to_string(), + ); + (None, None, 0, None, Err(e.to_string())) + } + }; Self { id, @@ -140,6 +145,7 @@ impl TcpConnectionContext { claims, destination_ip, destination_port, + address_family, sender, logger, } diff --git a/proxy_agent/src/proxy/proxy_server.rs b/proxy_agent/src/proxy/proxy_server.rs index d8a81f02..a40ba44f 100644 --- a/proxy_agent/src/proxy/proxy_server.rs +++ b/proxy_agent/src/proxy/proxy_server.rs @@ -907,6 +907,11 @@ impl ProxyServer { port: http_connection_context .tcp_connection_context .destination_port, + addressFamily: http_connection_context + .tcp_connection_context + .address_family + .map_or("IPv4", |family| family.as_str()) + .to_string(), responseStatus: response_status.to_string(), elapsedTime: elapsed_time.as_millis(), errorDetails: error_details, diff --git a/proxy_agent/src/proxy/proxy_summary.rs b/proxy_agent/src/proxy/proxy_summary.rs index ff823d7d..379bd5a8 100644 --- a/proxy_agent/src/proxy/proxy_summary.rs +++ b/proxy_agent/src/proxy/proxy_summary.rs @@ -9,6 +9,10 @@ use std::path::PathBuf; use proxy_agent_shared::proxy_agent_aggregate_status::ProxyConnectionSummary; use serde_derive::{Deserialize, Serialize}; +fn default_address_family() -> String { + "IPv4".to_string() +} + #[derive(Serialize, Deserialize, Clone)] #[allow(non_snake_case)] pub struct ProxySummary { @@ -19,6 +23,8 @@ pub struct ProxySummary { pub clientPort: u16, pub ip: String, pub port: u16, + #[serde(default = "default_address_family")] + pub addressFamily: String, pub userId: u64, pub userName: String, pub userGroups: Vec, @@ -33,11 +39,12 @@ pub struct ProxySummary { impl ProxySummary { pub fn to_key_string(&self) -> String { format!( - "{} {} {} {} {} {} {}", + "{} {} {} {} {} {} {} {}", self.userName, self.clientIp, self.ip, self.port, + self.addressFamily, self.processFullPath.to_string_lossy(), self.processCmdLine, self.responseStatus @@ -52,6 +59,7 @@ impl From for ProxyConnectionSummary { userGroups: Some(proxy_summary.userGroups), ip: proxy_summary.ip, port: proxy_summary.port, + addressFamily: proxy_summary.addressFamily, processFullPath: Some(proxy_summary.processFullPath.to_string_lossy().to_string()), processCmdLine: proxy_summary.processCmdLine, responseStatus: proxy_summary.responseStatus, diff --git a/proxy_agent/src/redirector.rs b/proxy_agent/src/redirector.rs index e94f1ee1..e32525f0 100644 --- a/proxy_agent/src/redirector.rs +++ b/proxy_agent/src/redirector.rs @@ -77,6 +77,23 @@ pub use linux::BpfObject; #[cfg(windows)] pub use windows::BpfObject; +#[derive(Clone, Copy, Debug, Default, Deserialize, Eq, PartialEq, Serialize)] +#[repr(u32)] +pub enum AddressFamily { + #[default] + IPv4 = 4, + IPv6 = 6, +} + +impl AddressFamily { + pub fn as_str(self) -> &'static str { + match self { + Self::IPv4 => "IPv4", + Self::IPv6 => "IPv6", + } + } +} + #[derive(Serialize, Deserialize)] #[repr(C)] pub struct AuditEntry { @@ -85,6 +102,8 @@ pub struct AuditEntry { pub is_admin: i32, pub destination_ipv4: u32, // in network byte order pub destination_port: u16, // in network byte order + #[serde(default)] + pub address_family: AddressFamily, } impl AuditEntry { @@ -95,6 +114,7 @@ impl AuditEntry { is_admin: 0, destination_ipv4: 0, destination_port: 0, + address_family: AddressFamily::IPv4, } } diff --git a/proxy_agent/src/redirector/linux.rs b/proxy_agent/src/redirector/linux.rs index 36571e4e..954ee7e5 100644 --- a/proxy_agent/src/redirector/linux.rs +++ b/proxy_agent/src/redirector/linux.rs @@ -8,6 +8,7 @@ use crate::common::{ }; use crate::redirector::shared_ebpf::linux_types::{ destination_entry, sock_addr_audit_entry, sock_addr_audit_key, sock_addr_skip_process_entry, + AuditMapKey, AuditMapValue, }; use crate::redirector::{ip_to_string, AuditEntry}; use crate::shared_state::redirector_wrapper::RedirectorSharedState; @@ -136,14 +137,22 @@ impl BpfObject { } pub fn attach_cgroup_program(&mut self, cgroup2_root_path: PathBuf) -> Result<()> { - let program_name = "connect4"; + self.attach_cgroup_program_by_name(cgroup2_root_path.clone(), "connect4")?; + self.attach_cgroup_program_by_name(cgroup2_root_path, "connect6") + } + + fn attach_cgroup_program_by_name( + &mut self, + cgroup2_root_path: PathBuf, + program_name: &str, + ) -> Result<()> { match std::fs::File::open(cgroup2_root_path.clone()) { Ok(cgroup) => match self.0.program_mut(program_name) { Some(program) => match program.try_into() { Ok(p) => { let program: &mut CgroupSockAddr = p; match program.load() { - Ok(_) => logger::write("connect4 program loaded.".to_string()), + Ok(_) => logger::write(format!("{program_name} program loaded.")), Err(err) => { return Err(Error::Bpf(BpfErrorType::LoadBpfProgram( program_name.to_string(), @@ -191,13 +200,13 @@ impl BpfObject { } pub fn attach_kprobe_program(&mut self) -> Result<()> { - let program_name = "tcp_v4_connect"; + let program_name = "tcp_connect_probe"; match self.0.program_mut(program_name) { Some(program) => match program.try_into() { Ok(p) => { let program: &mut KProbe = p; match program.load() { - Ok(_) => logger::write("tcp_v4_connect program loaded.".to_string()), + Ok(_) => logger::write(format!("{program_name} program loaded.")), Err(err) => { return Err(Error::Bpf(BpfErrorType::LoadBpfProgram( program_name.to_string(), @@ -208,7 +217,7 @@ impl BpfObject { match program.attach("tcp_connect", 0) { Ok(link_id) => { logger::write(format!( - "tcp_v4_connect program attached with id {link_id:?}." + "{program_name} program attached with id {link_id:?}." )); } Err(err) => { @@ -239,19 +248,13 @@ impl BpfObject { pub fn lookup_audit(&self, source_port: u16) -> Result { let audit_map_name = "audit_map"; match self.0.map(audit_map_name) { - Some(map) => match HashMap::try_from(map) { + Some(map) => match HashMap::<&MapData, AuditMapKey, AuditMapValue>::try_from(map) { Ok(audit_map) => { let key = sock_addr_audit_key::from_source_port(source_port); match audit_map.get(&key.to_array(), 0) { Ok(value) => { let audit_value = sock_addr_audit_entry::from_array(value); - Ok(AuditEntry { - logon_id: audit_value.logon_id as u64, - process_id: audit_value.process_id, - is_admin: audit_value.is_root as i32, - destination_ipv4: audit_value.destination_ipv4, - destination_port: audit_value.destination_port as u16, - }) + Ok(audit_value.to_audit_entry()) } Err(err) => Err(Error::Bpf(BpfErrorType::MapLookupElem( source_port.to_string(), @@ -355,7 +358,7 @@ impl BpfObject { pub fn remove_audit_map_entry(&mut self, source_port: u16) -> Result<()> { let audit_map_name = "audit_map"; match self.0.map_mut(audit_map_name) { - Some(map) => match HashMap::<&mut MapData, [u32; 2], [u32; 5]>::try_from(map) { + Some(map) => match HashMap::<&mut MapData, AuditMapKey, AuditMapValue>::try_from(map) { Ok(mut audit_map) => { let key = sock_addr_audit_key::from_source_port(source_port); audit_map.remove(&key.to_array()).map_err(|err| { @@ -486,7 +489,9 @@ pub async fn update_hostga_redirect_policy( mod tests { use crate::common::config; use crate::common::constants; - use crate::redirector::shared_ebpf::linux_types::{sock_addr_audit_entry, sock_addr_audit_key}; + use crate::redirector::shared_ebpf::linux_types::{ + sock_addr_audit_entry, sock_addr_audit_key, AuditMapKey, AuditMapValue, + }; use aya::maps::HashMap; use proxy_agent_shared::misc_helpers; use std::env; @@ -572,11 +577,13 @@ mod tests { is_root: 1, destination_ipv4: 0x10813FA8, destination_port: 80, + address_family: crate::redirector::shared_ebpf::GPA_ADDRESS_FAMILY_IPV6, + reserved: 0, }; { // drop map_mut("audit_map") within this scope - let mut audit_map: HashMap<&mut aya::maps::MapData, [u32; 2], [u32; 5]> = - HashMap::<&mut aya::maps::MapData, [u32; 2], [u32; 5]>::try_from( + let mut audit_map: HashMap<&mut aya::maps::MapData, AuditMapKey, AuditMapValue> = + HashMap::<&mut aya::maps::MapData, AuditMapKey, AuditMapValue>::try_from( bpf.0.map_mut("audit_map").unwrap(), ) .unwrap(); @@ -604,6 +611,7 @@ mod tests { entry.destination_port as u32, value.destination_port, "destination_port is not equal" ); + assert_eq!(entry.address_family, crate::redirector::AddressFamily::IPv6); } Err(err) => { println!("lookup_audit_internal error: {}", err); diff --git a/proxy_agent/src/redirector/shared_ebpf.rs b/proxy_agent/src/redirector/shared_ebpf.rs index 15bee538..eb4080d7 100644 --- a/proxy_agent/src/redirector/shared_ebpf.rs +++ b/proxy_agent/src/redirector/shared_ebpf.rs @@ -68,6 +68,8 @@ pub type destination_entry = _destination_entry; pub const IPPROTO_TCP: u32 = 6; #[allow(dead_code)] pub const IPPROTO_UDP: u32 = 17; +pub const GPA_ADDRESS_FAMILY_IPV4: u32 = 4; +pub const GPA_ADDRESS_FAMILY_IPV6: u32 = 6; #[repr(C)] pub struct sock_addr_skip_process_entry { @@ -95,6 +97,8 @@ pub struct sock_addr_audit_key { pub protocol: u32, pub source_port: u32, } +pub type AuditMapKey = + [u32; std::mem::size_of::() / std::mem::size_of::()]; #[allow(dead_code)] impl sock_addr_audit_key { #[cfg(windows)] @@ -113,11 +117,11 @@ impl sock_addr_audit_key { } } - pub fn to_array(&self) -> [u32; 2] { + pub fn to_array(&self) -> AuditMapKey { [self.protocol, self.source_port] } - pub fn from_array(array: [u32; 2]) -> Self { + pub fn from_array(array: AuditMapKey) -> Self { sock_addr_audit_key { protocol: array[0], source_port: array[1], @@ -133,7 +137,11 @@ pub struct sock_addr_audit_entry { pub is_root: u32, pub destination_ipv4: u32, pub destination_port: u32, + pub address_family: u32, + pub reserved: u32, } +pub type AuditMapValue = + [u32; std::mem::size_of::() / std::mem::size_of::()]; impl sock_addr_audit_entry { pub fn empty() -> Self { sock_addr_audit_entry { @@ -142,27 +150,33 @@ impl sock_addr_audit_entry { is_root: 0, destination_ipv4: 0, destination_port: 0, + address_family: GPA_ADDRESS_FAMILY_IPV4, + reserved: 0, } } - pub fn from_array(array: [u32; 5]) -> Self { + pub fn from_array(array: AuditMapValue) -> Self { sock_addr_audit_entry { logon_id: array[0], process_id: array[1], is_root: array[2], destination_ipv4: array[3], destination_port: array[4], + address_family: array[5], + reserved: array[6], } } #[allow(dead_code)] - pub fn to_array(&self) -> [u32; 5] { + pub fn to_array(&self) -> AuditMapValue { [ self.logon_id, self.process_id, self.is_root, self.destination_ipv4, self.destination_port, + self.address_family, + self.reserved, ] } @@ -173,6 +187,11 @@ impl sock_addr_audit_entry { is_admin: self.is_root as i32, destination_ipv4: self.destination_ipv4, destination_port: self.destination_port as u16, + address_family: if self.address_family == GPA_ADDRESS_FAMILY_IPV6 { + crate::redirector::AddressFamily::IPv6 + } else { + crate::redirector::AddressFamily::IPv4 + }, } } } @@ -204,6 +223,7 @@ impl sock_addr_audit_entry_legacy { is_admin: self.is_admin, destination_ipv4: self.destination_ipv4, destination_port: self.destination_port, + address_family: crate::redirector::AddressFamily::IPv4, } } } @@ -361,7 +381,8 @@ impl AuditValueEntry { #[cfg(not(windows))] pub mod linux_types { pub use super::{ - destination_entry, sock_addr_audit_entry, sock_addr_audit_key, sock_addr_skip_process_entry, + destination_entry, sock_addr_audit_entry, sock_addr_audit_key, + sock_addr_skip_process_entry, AuditMapKey, AuditMapValue, }; } @@ -433,6 +454,8 @@ mod tests { is_root: 1, destination_ipv4: 4, destination_port: 5, + address_family: GPA_ADDRESS_FAMILY_IPV6, + reserved: 0, }; let rebuilt = sock_addr_audit_entry::from_array(canonical.to_array()); @@ -442,6 +465,7 @@ mod tests { assert_eq!(rebuilt.is_root, canonical.is_root); assert_eq!(rebuilt.destination_ipv4, canonical.destination_ipv4); assert_eq!(rebuilt.destination_port, canonical.destination_port); + assert_eq!(rebuilt.address_family, canonical.address_family); } #[test] @@ -452,6 +476,8 @@ mod tests { is_root: 1, destination_ipv4: 0x0102_0304, destination_port: u32::from(8080u16.to_be()), + address_family: GPA_ADDRESS_FAMILY_IPV6, + reserved: 0, }; let audit = canonical.to_audit_entry(); @@ -460,6 +486,7 @@ mod tests { assert_eq!(audit.is_admin, 1); assert_eq!(audit.destination_ipv4, 0x0102_0304); assert_eq!(audit.destination_port, 8080u16.to_be()); + assert_eq!(audit.address_family, crate::redirector::AddressFamily::IPv6); } #[test] @@ -572,6 +599,8 @@ mod tests { is_root: 1, destination_ipv4: 0x0A00_0001, destination_port: u32::from(443u16.to_be()), + address_family: GPA_ADDRESS_FAMILY_IPV6, + reserved: 0, }); let audit = new_entry.to_audit_entry().expect("New should convert"); assert_eq!(audit.logon_id, 11); @@ -579,6 +608,7 @@ mod tests { assert_eq!(audit.is_admin, 1); assert_eq!(audit.destination_ipv4, 0x0A00_0001); assert_eq!(audit.destination_port, 443u16.to_be()); + assert_eq!(audit.address_family, crate::redirector::AddressFamily::IPv6); // Legacy variant converts directly. let legacy_entry = AuditValueEntry::Legacy(sock_addr_audit_entry_legacy { @@ -604,6 +634,8 @@ mod tests { is_root: 1, destination_ipv4: 0x0808_0808, destination_port: u32::from(53u16.to_be()), + address_family: GPA_ADDRESS_FAMILY_IPV6, + reserved: 0, }; let mut unknown_new = AuditValueEntry::empty(AuditValueEntry::VALUE_SIZE_NEW + 1); fill_unknown_buffer( @@ -622,6 +654,7 @@ mod tests { assert_eq!(audit.is_admin, 1); assert_eq!(audit.destination_ipv4, 0x0808_0808); assert_eq!(audit.destination_port, 53u16.to_be()); + assert_eq!(audit.address_family, crate::redirector::AddressFamily::IPv6); // Unknown variant decodes a legacy layout from raw bytes. let legacy = sock_addr_audit_entry_legacy { diff --git a/proxy_agent/src/redirector/windows.rs b/proxy_agent/src/redirector/windows.rs index e1b214fb..ad167ae5 100644 --- a/proxy_agent/src/redirector/windows.rs +++ b/proxy_agent/src/redirector/windows.rs @@ -14,10 +14,13 @@ use std::ptr; use windows_sys::Win32::Networking::WinSock; /// Wrapper for eBPF object and link -/// This struct holds pointers to the eBPF object and its associated link. +/// This struct holds a pointer to the eBPF object and its associated links. /// Start from ebpf-for-windows v1.0.0-rc, eBPF programs need to keep the link alive /// to ensure the eBPF program remains attached. -pub struct BpfObject(pub *mut bpf_obj::bpf_object, pub *mut bpf_obj::ebpf_link_t); +pub struct BpfObject( + pub *mut bpf_obj::bpf_object, + pub Vec<*mut bpf_obj::ebpf_link_t>, +); // Safety: bpf_object, which is a reference to an eBPF object, has no dependencies on thread-local storage and can // safely be sent to another thread. This is not explicitly documented in the Windows eBPF library, but the library does // document it aims to be source-compatible with libbpf[0]. Note that synchronization is required to share this object diff --git a/proxy_agent/src/redirector/windows/bpf_prog.rs b/proxy_agent/src/redirector/windows/bpf_prog.rs index 3a0fb5e9..119eeaab 100644 --- a/proxy_agent/src/redirector/windows/bpf_prog.rs +++ b/proxy_agent/src/redirector/windows/bpf_prog.rs @@ -22,10 +22,7 @@ impl BpfObject { } pub fn new() -> Self { - Self( - std::ptr::null::().cast_mut(), - std::ptr::null::().cast_mut(), - ) + Self(std::ptr::null::().cast_mut(), Vec::new()) } /** @@ -94,7 +91,7 @@ impl BpfObject { /** Routine Description: - This routine attach authorize_connect4 to bpf. + This routine attaches the IPv4 and IPv6 connect programs to bpf. Arguments: @@ -106,8 +103,12 @@ impl BpfObject { if self.is_null() { return Err(Error::Bpf(BpfErrorType::NullBpfObject)); } - let program_name = "authorize_connect4"; - let connect4_program = match bpf_object__find_program_by_name(self.0, program_name) { + self.attach_bpf_prog_by_name("authorize_connect4")?; + self.attach_bpf_prog_by_name("authorize_connect6") + } + + fn attach_bpf_prog_by_name(&mut self, program_name: &str) -> Result<()> { + let program = match bpf_object__find_program_by_name(self.0, program_name) { Ok(p) => { logger::write_information(format!("Found {program_name} program.")); p @@ -119,7 +120,7 @@ impl BpfObject { ))); } }; - if connect4_program.is_null() { + if program.is_null() { return Err(Error::Bpf(BpfErrorType::AttachBpfProgram( program_name.to_string(), "bpf_object__find_program_by_name return null".to_string(), @@ -129,9 +130,8 @@ impl BpfObject { let compartment_id = 1; let mut link: ebpf_link_t = ebpf_link_t::empty(); let mut link: *mut ebpf_link_t = &mut link as *mut ebpf_link_t; - //let link: *mut *mut ebpf_link_t = &mut link as *mut *mut ebpf_link_t; match ebpf_prog_attach( - connect4_program, + program, std::ptr::null(), &compartment_id as *const i32 as *const c_void, size_of_val(&compartment_id), @@ -144,10 +144,8 @@ impl BpfObject { format!("ebpf_prog_attach return with error code '{r}'"), ))); } - logger::write_information( - "Success attached authorize_connect4 program.".to_string(), - ); - self.1 = link; + logger::write_information(format!("Successfully attached {program_name} program.")); + self.1.push(link); } Err(e) => { return Err(Error::Bpf(BpfErrorType::AttachBpfProgram( @@ -235,16 +233,17 @@ impl BpfObject { } self.0 = std::ptr::null::().cast_mut(); - if self.1.is_null() { - return; - } - if let Err(e) = bpf_link_disconnect(self.1) { - logger::write_error(format!("bpf_link_disconnect with error: {e}")); - } - if let Err(e) = bpf_link_destroy(self.1) { - logger::write_error(format!("bpf_link_destroy with error: {e}")); + for link in self.1.drain(..) { + if link.is_null() { + continue; + } + if let Err(e) = bpf_link_disconnect(link) { + logger::write_error(format!("bpf_link_disconnect with error: {e}")); + } + if let Err(e) = bpf_link_destroy(link) { + logger::write_error(format!("bpf_link_destroy with error: {e}")); + } } - self.1 = std::ptr::null::().cast_mut(); } /** diff --git a/proxy_agent/src/shared_state/connection_summary_wrapper.rs b/proxy_agent/src/shared_state/connection_summary_wrapper.rs index 4d94a57a..d025c1a8 100644 --- a/proxy_agent/src/shared_state/connection_summary_wrapper.rs +++ b/proxy_agent/src/shared_state/connection_summary_wrapper.rs @@ -265,6 +265,7 @@ mod tests { clientPort: 6080, ip: "127.0.0.1".to_string(), port: 8080, + addressFamily: "IPv4".to_string(), userId: 999, userName: "user1".to_string(), userGroups: vec!["group1".to_string()], @@ -286,6 +287,15 @@ mod tests { assert_eq!(1, get_all_connection_summary.len()); assert_eq!(1, get_all_connection_summary[0].count); + let mut ipv6_summary = connection_summary.clone(); + ipv6_summary.addressFamily = "IPv6".to_string(); + assert_ne!( + connection_summary.to_key_string(), + ipv6_summary.to_key_string() + ); + let telemetry_json = serde_json::to_string(&ipv6_summary).unwrap(); + assert!(telemetry_json.contains(r#""addressFamily":"IPv6""#)); + let failed_connection_summary = ProxySummary { id: 2, method: "GET".to_string(), @@ -294,6 +304,7 @@ mod tests { clientPort: 6080, ip: "127.0.0.1".to_string(), port: 8080, + addressFamily: "IPv4".to_string(), userId: 999, userName: "user1".to_string(), userGroups: vec!["group1".to_string()], diff --git a/proxy_agent_extension/src/service_main.rs b/proxy_agent_extension/src/service_main.rs index be725082..7205b70f 100644 --- a/proxy_agent_extension/src/service_main.rs +++ b/proxy_agent_extension/src/service_main.rs @@ -1088,6 +1088,7 @@ mod tests { userName: "test".to_string(), ip: "test".to_string(), port: 1, + addressFamily: "IPv4".to_string(), processCmdLine: "test".to_string(), responseStatus: "test".to_string(), count: 1, diff --git a/proxy_agent_shared/src/proxy_agent_aggregate_status.rs b/proxy_agent_shared/src/proxy_agent_aggregate_status.rs index 70def9ab..ce744931 100644 --- a/proxy_agent_shared/src/proxy_agent_aggregate_status.rs +++ b/proxy_agent_shared/src/proxy_agent_aggregate_status.rs @@ -59,12 +59,18 @@ pub struct ProxyAgentStatus { pub proxyConnectionsCount: u128, } +fn default_address_family() -> String { + "IPv4".to_string() +} + #[derive(Serialize, Deserialize)] #[allow(non_snake_case)] pub struct ProxyConnectionSummary { pub userName: String, pub ip: String, pub port: u16, + #[serde(default = "default_address_family")] + pub addressFamily: String, pub processCmdLine: String, pub responseStatus: String, pub count: u64, @@ -79,6 +85,7 @@ impl Clone for ProxyConnectionSummary { userGroups: self.userGroups.clone(), ip: self.ip.clone(), port: self.port, + addressFamily: self.addressFamily.clone(), processFullPath: self.processFullPath.clone(), processCmdLine: self.processCmdLine.clone(), responseStatus: self.responseStatus.clone(), diff --git a/shared-ebpf/include/gpa_audit_event.h b/shared-ebpf/include/gpa_audit_event.h index 667cd36f..5368acd8 100644 --- a/shared-ebpf/include/gpa_audit_event.h +++ b/shared-ebpf/include/gpa_audit_event.h @@ -12,6 +12,9 @@ #pragma once +#define GPA_ADDRESS_FAMILY_IPV4 4 +#define GPA_ADDRESS_FAMILY_IPV6 6 + // IP address - union allows IPv4 (first element) or IPv6 (all 4 elements) // Size: 16 bytes (4 x u32) - matches Rust _ip_address { ip: [u32; 4] } struct gpa_ip_address @@ -42,7 +45,7 @@ struct gpa_audit_key }; // Canonical audit event entry - the record stored in the audit map -// Size: 20 bytes - matches Rust sock_addr_audit_entry -> [u32; 5] +// Size: 28 bytes - matches Rust sock_addr_audit_entry -> [u32; 7] // // NOTE: Field names use Linux semantics (logon_id=uid, is_root). The Windows // side maps these to its own naming (logon_id, is_admin) at user-space. @@ -54,6 +57,8 @@ struct gpa_audit_event __u32 is_root; // 1 if root/admin, 0 otherwise __u32 destination_ipv4; // Destination IPv4 address __u32 destination_port; // Destination port (stored as u32) + __u32 address_family; // GPA_ADDRESS_FAMILY_IPV4 or GPA_ADDRESS_FAMILY_IPV6 + __u32 reserved; }; // Skip process entry - processes in this map bypass audit/redirect @@ -64,7 +69,7 @@ struct gpa_skip_process_entry }; // Local address entry - tracks current connection state in the local_map -// Size: 24 bytes (6 x u32) +// Size: 32 bytes (8 x u32) struct gpa_sock_addr_local_entry { __u32 logon_id; // uid @@ -73,6 +78,8 @@ struct gpa_sock_addr_local_entry __u32 destination_ipv4; __u32 destination_port; __u32 protocol; + __u32 address_family; + __u32 reserved; }; // Compile-time layout assertions to guarantee binary compatibility with Rust loader. @@ -80,6 +87,6 @@ struct gpa_sock_addr_local_entry _Static_assert(sizeof(struct gpa_ip_address) == 16, "ip_address must be 16 bytes ([u32; 4])"); _Static_assert(sizeof(struct gpa_destination_entry) == 24, "destination_entry must be 24 bytes ([u32; 6])"); _Static_assert(sizeof(struct gpa_audit_key) == 8, "audit_key must be 8 bytes ([u32; 2])"); -_Static_assert(sizeof(struct gpa_audit_event) == 20, "audit_event must be 20 bytes ([u32; 5])"); +_Static_assert(sizeof(struct gpa_audit_event) == 28, "audit_event must be 28 bytes ([u32; 7])"); _Static_assert(sizeof(struct gpa_skip_process_entry) == 4, "skip_process_entry must be 4 bytes ([u32; 1])"); -_Static_assert(sizeof(struct gpa_sock_addr_local_entry) == 24, "sock_addr_local_entry must be 24 bytes ([u32; 6])"); +_Static_assert(sizeof(struct gpa_sock_addr_local_entry) == 32, "sock_addr_local_entry must be 32 bytes ([u32; 8])");