nftables: support an nftables-based egress-firewall sidecar under runsc (OpenSandbox) — use case + tracking PRs

[ferponse]

Why we need this (use case)

OpenSandbox runs untrusted, AI-generated code in ephemeral, per-request pods on Kubernetes. gVisor (GKE Sandbox) is the isolation runtime we want for exactly this "safely run untrusted code" scenario.

Each sandbox pod ships an egress-firewall sidecar (runs with CAP_NET_ADMIN) that enforces an outbound network policy so the untrusted workload can only reach an allowlist. It programs the dataplane entirely with nftables:

• a nat / output DNS redirect to a local DNS proxy (redirect to :15353), with a meta mark bypass for the proxy's own upstream queries;
• a transparent-HTTP redirect for MITM inspection (meta skuid != <uid>, tcp dport {80,443}, redirect to :<port>);
• an inet filter chain on the output hook implementing a domain→IP allowlist: interval sets (CIDR allow/deny), timeout sets (dynamically-resolved IPs that expire), ct state established,related accept, meta mark accept, oifname "lo" accept, and ip/ip6 daddr @set lookups.

Under runsc this ruleset could not be installed because several nft features were unimplemented, so the egress sidecar could not run under gVisor at all.

It works under Kata, but we want gVisor

This exact setup works today under Kata Containers — Kata gives each pod a full Linux kernel, so the egress sidecar's nat/REDIRECT rule works — but Kata puts a full micro-VM under every sandbox, i.e. much higher cold-start latency and per-pod memory/density overhead, which is expensive for ephemeral, per-request sandboxes. We want gVisor's lighter-weight, syscall-level isolation instead.

OpenSandbox documents this gVisor-vs-Kata situation:

• Egress sidecar incompatible with gVisor, with Kata as the documented workaround: docs/guides/secure-container.md §"Egress Sidecar Incompatible with gVisor" (same doc also has the runtime cold-start / overhead comparison table)
• Network-isolation rationale — the egress sidecar's nat-table REDIRECT works under runc and all Kata variants (kata-qemu/kata-clh/kata-fc) but not gVisor: docs/architecture/network-isolation.md §"Runtime Compatibility"
• Kata deployment example: docs/examples/aks-kata.md

(Those docs attribute the gVisor limitation to the iptables nat table / gvisor#170. The PRs below provide the native-nftables path — redir plus the set backends — so the egress can run under gVisor without relying on the iptables nat table at all.)

What was missing, and the PRs

Running the egress's exact ruleset under runsc --TESTONLY-nftables (Docker + nft), the gaps were as follows (some pieces — ct state, oifname, meta skuid, set lookups, verdicts — already worked):

Gap	PR
meta mark load	#13797
interval sets (CIDR allow/deny)	#13799
timeout sets (expiring dynamic allow)	#13800
redir / REDIRECT (DNS + MITM redirect)	#13801

Each PR includes unit tests plus end-to-end validation under runsc. The redir change is additionally validated functionally: with a redirect rule installed, a datagram sent to 8.8.8.8:53 is delivered to a local listener bound to 127.0.0.1:15353.

With these four, the entire nft policy engine the egress needs works under gVisor — no iptables-nft/xt-compat is required. The matching OpenSandbox change that makes the egress emit native nft for the DNS and MITM redirects (instead of shelling out to iptables) is opensandbox-group/OpenSandbox#1374 — kept in draft until these gVisor PRs land and a runsc release ships with them.

Suggested merge order

• Support meta mark load in nftables #13797 (meta mark) — independent (touches nft_meta.go / nft_metaload.go); can merge any time.
• Support interval sets in nftables #13799 (interval sets) → Support timeout sets in nftables #13800 (timeout sets) → Support the redir (REDIRECT) expression in nftables #13801 (redir) — these touch the shared pkg/tcpip/nftables package and are stacked in that order. Please merge in sequence; each is rebased onto master after the previous lands.

Notes / out of scope

• Set-element deletion (NFT_MSG_DELSETELEM, NFTables: Support NFT_MSG_DELSETELEM and NFT_MSG_DESTROYSETELEM. #13789) is not required by this use case — the egress relies on set timeouts for expiry, not explicit deletes — and is orthogonal.
• Relates to the umbrella Support NETLINK_NETFILTER sockets #11778 (NETLINK_NETFILTER support) and nftables bytecode interpreter #10510.

[ferponse]

Reopening with a clearer write-up: the OpenSandbox use case, the set of PRs, and the suggested merge order.

[nybidari]

Thanks @ferponse for providing the context and the detailed breakdown of the use case!
We are happy to review the PRs.

cc: @parth-opensrc