Observability Wiki

Profiling

eBPF-based Profiling Explained

How eBPF lets a profiler sample every process on a Linux host without touching application code or installing per-language agents.

Last updated

eBPF (extended Berkeley Packet Filter) is a Linux kernel technology that runs small, verified programs inside the kernel in response to events — a network packet arriving, a system call, a scheduler tick, or a timer firing — without requiring a kernel module or a kernel recompile. It was originally built for high-performance packet filtering, but the same mechanism turned out to be an ideal foundation for observability tooling, including continuous profiling.

An eBPF-based profiler exploits this to sample every process running on a host — regardless of the language it’s written in — from a single agent running outside those processes.

Why it matters

  • No per-language agents or code changes. A JVM needs a Java agent, a Python process needs py-spy or similar, a Go binary needs its own runtime hooks — traditionally, every language required its own profiler. An eBPF profiler attaches at the kernel level and can sample any process on the machine, whatever it’s written in, with zero modification to the application.
  • Whole-system visibility. Because the agent runs once per host rather than once per process, it profiles everything on that machine simultaneously — application code, sidecars, system daemons — giving a single consistent view of where a whole node’s CPU actually goes.
  • Very low overhead. eBPF programs run in a lightweight, verified in-kernel sandbox rather than injecting instrumentation into the target process, so well-implemented eBPF profilers (Parca Agent is commonly cited) add well under 1% CPU overhead, which is what makes running them continuously and fleet-wide practical.
  • No restarts required. Because it doesn’t inject anything into the target process, an eBPF profiler can be attached and detached from a running fleet without redeploying or restarting a single workload.

How it works

  • Attaching to a perf event. The profiler registers an eBPF program against the kernel’s perf_event subsystem, configured to fire at a fixed frequency (commonly 100 times per second per CPU core).
  • In-kernel stack walking. On each timer tick, the eBPF program captures whatever is executing on that CPU at that instant and walks the call stack, recording raw frame addresses (instruction pointers) into a kernel map — this all happens in-kernel, without stopping or attaching a debugger to the target process.
  • User-space symbolication. Raw stack samples are just memory addresses; a user-space component drains them from the kernel map and resolves each address back to a function name and source location using the target binary’s debug symbols (or, for interpreted/JIT’d languages, additional runtime-specific metadata).
  • Aggregation into a profile format. Matching stacks are counted and merged into a profile — typically pprof format — which is periodically pushed to a profiling backend such as Parca or a hosted equivalent like Polar Signals Cloud, where it can be queried over time and rendered as a flame graph.

Current limitations

  • Kernel version requirements. Advanced eBPF features (particularly efficient stack-walking helpers) require a reasonably modern kernel; older distributions or heavily restricted container runtimes may not expose the needed capabilities, and some environments require elevated privileges to load eBPF programs at all.
  • Symbolication is hard for JIT’d and interpreted languages. Compiled languages like Go, Rust, or C++ ship stable symbol tables that are straightforward to resolve. Languages with a JIT (the JVM, Node.js’s V8) or that primarily execute an interpreter loop (Python, Ruby) generate machine code at runtime or spend most samples inside a generic interpreter frame, so recovering a meaningful application-level function name typically needs extra runtime instrumentation or frame-pointer support layered on top of the base eBPF sampling.
  • Frame pointers. Reliable stack walking depends on the target binary preserving frame pointers (or, more recently, kernel support for DWARF-based unwinding); binaries compiled with frame pointers omitted can produce incomplete or broken stacks.

Because it requires no per-language integration, eBPF profiling is often the fastest way to get baseline visibility across a mixed-language fleet, complementing OpenTelemetry-based instrumentation rather than replacing language-specific profilers where the deepest symbol accuracy is needed.

Related tools