ebpf-scaler-prototype

A single-node Kubernetes autoscaler prototype. For every pod in a target namespace it attaches an XDP program to the pod's veth interface (discovered via the Pods API plus the node routing table), measures per-pod traffic rates, and scales the owning Deployments toward a per-pod target using the HPA formula desired = ceil(current * avg_rate / target).

How it works

Each pod gets its own instance of the xdp_counter BPF program with a private per-CPU map counting packets and bytes. Where it attaches depends on the measured direction:

• --direction ingress (default) — traffic into the pod. XDP only sees packets received on its interface, so the program is attached to eth0 inside the pod's network namespace. The pod's netns is found by scanning the CNI-created named namespaces (/var/run/netns/cni-*) for the interface holding the pod IP, then the attach syscall runs on a thread setns()'d into it.
• --direction egress — traffic out of the pod. Attached to the host-side caliXXXX veth, whose RX path is the pod's TX.

Every --interval-secs the control loop:

1. lists Running pods in the namespace scheduled on this node, resolves each pod's host-side veth from the node's /32 routes (/proc/net/route — Calico programs one per local pod), attaches to new pods and detaches from deleted ones;
2. reads each pod's counters and computes a packets/s or bytes/s rate;
3. maps pods to Deployments via Pod → ReplicaSet → Deployment owner references;
4. per Deployment, compares the average per-pod rate to --target-per-pod (with the HPA's 10% tolerance) and patches .spec.replicas, subject to --min-replicas / --max-replicas and a --cooldown-secs between scale operations.

Building

# Libraries and tools
sudo apt-get update -y
sudo apt-get install -y libbpf-dev cmake clang llvm

# Generate vmlinux.h using bpftool (once per kernel)
bpftool btf dump file /sys/kernel/btf/vmlinux format c > bpf/vmlinux.h

# Build the eBPF ELF
clang -g -O2 -target bpf -c bpf/xdp_counter.bpf.c -o bpf/xdp_counter.bpf.o

# Build the autoscaler
cargo build --release

Running

Must run on the cluster node hosting the pods (single-node prototype), as root (BPF load/attach + entering pod network namespaces), with a kubeconfig that can list pods, get replicasets, and get/patch deployments/scale in the target namespace.

# Scale everything in "demo" on pod ingress packet rate, targeting 5k pps/pod
sudo -E RUST_LOG=info ./target/release/ebpf-scaler-prototype \
    --namespace demo --target-per-pod 5000

# Scale one Deployment on egress byte rate, 1 MiB/s per pod, up to 20 replicas
sudo -E RUST_LOG=info ./target/release/ebpf-scaler-prototype \
    --namespace demo --deployment web --metric bytes --direction egress \
    --target-per-pod 1048576 --max-replicas 20

# Observe decisions without touching replica counts
sudo -E RUST_LOG=info ./target/release/ebpf-scaler-prototype \
    --namespace demo --target-per-pod 5000 --dry-run

--help lists all flags (interval, cooldown, min/max replicas, node name, BPF object path, netns directory).

Usage

Autoscales Deployments on per-pod traffic rates measured by XDP programs attached to Calico pod interfaces

Usage: ebpf-scaler-prototype [OPTIONS] --namespace <NAMESPACE> --target-per-pod <TARGET_PER_POD>

Options:
  -n, --namespace <NAMESPACE>
          Namespace whose pods are instrumented and whose Deployments are scaled

  -d, --deployment <DEPLOYMENT>
          Only scale this Deployment (default: every Deployment in the namespace)

  -m, --metric <METRIC>
          Metric driving the scaling decision

          Possible values:
          - packets: Scale on packets per second
          - bytes:   Scale on bytes per second
          
          [default: packets]

      --direction <DIRECTION>
          Traffic direction to measure

          Possible values:
          - ingress: Traffic received by the pod (XDP on eth0 inside the pod netns)
          - egress:  Traffic sent by the pod (XDP on the host-side cali interface)
          
          [default: ingress]

  -t, --target-per-pod <TARGET_PER_POD>
          Target average per-pod rate (packets/s or bytes/s depending on --metric)

      --min-replicas <MIN_REPLICAS>
          [default: 1]

      --max-replicas <MAX_REPLICAS>
          [default: 10]

      --interval-secs <INTERVAL_SECS>
          Seconds between metric samples / scaling decisions
          
          [default: 10]

      --cooldown-secs <COOLDOWN_SECS>
          Minimum seconds between two scale operations on the same Deployment
          
          [default: 60]

      --node-name <NODE_NAME>
          Kubernetes node this process runs on (default: $NODE_NAME or hostname)

      --bpf-object <BPF_OBJECT>
          Compiled XDP counter object
          
          [default: bpf/xdp_counter.bpf.o]

      --netns-dir <NETNS_DIR>
          Directory of named network namespaces created by the CNI runtime
          
          [default: /var/run/netns]

      --dry-run
          Log scaling decisions without patching Deployments

  -h, --help
          Print help (see a summary with '-h')

  -V, --version
          Print version

Notes and limitations

• Single-node only: XDP must be attached on the node where the veth lives. Multi-node would need this as a DaemonSet plus rate aggregation.
• XDP is attached in SKB (generic) mode, since native veth XDP requires driver-side support on both peers.
• Ingress mode needs named netns files under /var/run/netns (containerd's CNI does this; override with --netns-dir).
• Counters are per-CPU and summed at read time; rates are deltas between consecutive samples, so a freshly attached pod contributes from its second sample onward.

Debugging the BPF object

llvm-objdump -d bpf/xdp_counter.bpf.o
llvm-readelf -S bpf/xdp_counter.bpf.o