CPU vs. Memory vs. Wall-clock Profiling

“Profiling” is not one measurement — it’s a family of sampling modes that each answer a different question about where a program’s resources go. The three most common are CPU (on-CPU time), memory (allocations), and wall-clock (elapsed real time, including time spent waiting). Picking the wrong mode for a given problem is one of the most common ways engineers conclude “the profiler shows nothing interesting” when the bottleneck is actually invisible to the mode they used.

Why it matters

• On-CPU profiling only sees threads that are actually running. A CPU profiler samples the call stack of threads in a runnable state on a core; a thread that’s blocked waiting on a lock, a socket read, or a database round-trip is not scheduled on a CPU and simply never gets sampled. This makes CPU profiling excellent for compute-bound bottlenecks and blind to I/O-bound ones.
• Memory (allocation) profiling tracks a different resource entirely. Instead of sampling what’s executing, an allocation profiler records where objects are allocated and how much heap pressure each call site generates — the top frame in an allocation profile is the type of object allocated, not necessarily where the most CPU time went. This is the right tool for chasing high garbage-collection pressure or unexpected memory growth, and largely useless for finding a slow, allocation-free loop.
• Wall-clock profiling closes the gap CPU profiling leaves. A wall-clock (sometimes “off-CPU”) profiler samples all threads at a fixed interval regardless of their state — running, sleeping, or blocked — so a thread stuck waiting on a lock or a network call shows up in the profile just as prominently as one burning CPU. This is what reveals I/O-bound and contention-bound latency that on-CPU sampling misses entirely.
• A profiler that only does on-CPU sampling can report a service as “fine” while it’s actually stalled on a downstream dependency. If most of a request’s latency is spent blocked on a slow database call, a CPU-only profile of that request will show almost nothing — the thread wasn’t using CPU during the slow part, so there’s nothing to sample. Only wall-clock or off-CPU analysis surfaces that time.

Key concepts

• On-CPU / CPU-time profiling. Samples are taken only while a thread is actively executing on a core. This is the classic, cheapest form of profiling and is the right choice for CPU-bound workloads: tight loops, serialization, compression, hashing, anything where the code itself is the cost.
• Wall-clock (elapsed time) profiling. Samples all threads on a timer regardless of scheduling state, capturing both on-CPU and off-CPU (blocked, sleeping, waiting) time. It’s the right choice when total request latency matters more than CPU consumption specifically — for example, diagnosing why a request takes 800ms when the service is barely using any CPU.
• Off-CPU profiling is a closely related, narrower variant that specifically targets time spent not running — blocked on I/O, mutexes, or scheduler waits — often paired with a separate on-CPU profile so the two can be compared directly rather than blended into one wall-clock view.
• Memory / allocation profiling. Instead of time, this mode samples allocation events (and sometimes tracks live heap contents rather than allocation rate), attributing bytes allocated back to the call site responsible. Tools like async-profiler expose this as a distinct profiling event separate from CPU or wall-clock sampling, and some profilers additionally support lock/contention profiling as a fourth, related mode.
• Modes are usually configured, not automatic. Profilers such as async-profiler let you choose the event to sample — cpu, wall, alloc, lock — and some support capturing more than one simultaneously; a continuous profiling backend like Pyroscope or Parca will typically ingest whichever mode(s) the agent is configured to emit, tagged so they can be viewed as separate flame graphs.

The practical takeaway: match the profiling mode to the symptom. A high CPU utilization alert calls for on-CPU profiling; unexplained memory growth calls for allocation profiling; a slow endpoint with low CPU usage is a strong signal to reach for wall-clock or off-CPU profiling instead of assuming the on-CPU profile that shows “nothing” is the full story.