eBPF: Observability Without the Sidecar Tax

For years the answer to "instrument this service" was a sidecar: a little proxy or agent sharing the pod's network namespace, capturing traffic, forwarding metrics. It worked, and it cost — extra latency hops, memory per pod, image pulls, and a startup-order dependency that made rollouts exciting.

Then eBPF stopped being a research curiosity and became the boring default. If you're running a modern Kubernetes cluster in 2026 and not using it somewhere, you're probably paying the sidecar tax without knowing it.

The one-paragraph version

eBPF (extended Berkeley Packet Filter) lets you run small, verified programs inside the kernel — on syscall entry, on network events, on scheduler activity — without touching kernel source or loading modules. The kernel's verifier proves the program can't crash or loop forever before it runs, then it executes with startling efficiency, feeding data to user space through maps and rings.

Translation: you can observe and even enforce behavior system-wide from one program per node, instead of one agent per pod.

What it replaced on my clusters

Metrics and request flow. Instead of instrumenting every service and proxying through a sidecar, node-level eBPF probes read the kernel's view of TCP connections, DNS queries, and HTTP (via uprobe/kprobe on the TLS libraries or socket filters). You get golden signals — request rate, error rate, latency buckets, per-connection byte counts — for every service, including the three nobody instrumented. Tools in this space (Cilium's Hubble, Pixie, various commercial agents) turn that into dashboards and flows maps with zero code changes.

Network policy enforcement. Cilium implements Kubernetes NetworkPolicy in eBPF rather than iptables. For big clusters this matters: iptables rule evaluation historically grew quadratically with policy count; eBPF programs are flat. Same YAML, fewer CPU spikes during churn.

Runtime security. Agents like Tetragon hook syscalls and audit events at kernel level: "this process spawned a shell inside a container that never shells out." Sidecar-based tools see the network; eBPF sees the syscalls, which is where container escapes announce themselves.

A quick taste of the raw tooling

Before the platforms wrap it up, it's worth touching the primitive. On a Linux box with kernel headers:

# count syscalls by process, top talkers in seconds
sudo bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @[comm] = count(); }'

That one-liner attaches a verified program to a kernel tracepoint and aggregates in-kernel — no agent fleet, no daemonset, no sidecar. You won't ship bpftrace scripts to production (you'll use the platforms), but five minutes with it builds the mental model everything else rests on.

The honest constraints

Kernel version and platform support. Modern features want kernel 5.8+; managed platforms now mostly oblige (EKS, GKE, and AKS all ship eBPF-capable node images), but check your nodes — and your on-prem kernel policy — before designing around it. Where it's unsupported, the fallback is still the old agents.

The verifier will reject your program if you write it carelessly. Bounded loops, validated pointer access — the verifier's strictness is the feature that makes kernel execution safe, but it's a real learning curve if you write eBPF by hand. (Most people won't; they'll consume it.)

It's not magic for application semantics. eBPF sees bytes and syscalls, not business logic. "Checkout is failing" still needs traces and logs with request context. eBPF is superb at plumbing-level truth — who's talking to whom, where latency lives in the socket stack, what DNS is doing — which is precisely what sidecars were overqualified for.

Portability between arch and kernels is good but not infinite; exotic distributions may lag. Budget a spike for your actual node image if it isn't a mainstream one.

How I'd adopt it

Start with one node pool. Install an eBPF-based network policy engine or observability agent alongside your existing stack, run both for two weeks, and compare: did the flow map find connections your service mesh didn't show? Did policy enforcement drop conntrack load? Did anyone notice the sidecars leave?

On our last migration, the sidecar memory saving across the cluster was around 15% of total allocatable — and the flows map found two services quietly calling a third-party API nobody had documented. The sidecars didn't tell us that, because they were per-pod and nobody had wired them into the map.

The kernel already knows everything happening on the box. eBPF is just the honest way to ask it.