How Google uses eBPF to boost RPC performance at scale

Google has revealed how it is using eBPF-based CPU scheduling to extract greater performance from its vast computing infrastructure, delivering measurable gains for key workloads while avoiding the risks traditionally associated with kernel modifications.

For organisations operating hyperscale infrastructure, even small improvements in system efficiency can translate into substantial cost savings and performance gains. According to Google, moving scheduling policies into programmable eBPF components has led to around a five percent increase in queries per second (QPS) for some of its core Remote Procedure Call (RPC) serving stacks, alongside double-digit throughput improvements for memory-intensive applications. The development highlights a broader trend in systems engineering: the drive to separate policy decisions from the underlying mechanisms that implement them. By doing so, organisations can optimise workloads more rapidly without rewriting large sections of operating system code.

The limits of traditional CPU schedulers

Modern Linux kernels include sophisticated CPU schedulers designed to balance competing workloads across multiple processors. However, these schedulers are intentionally general-purpose. That approach works well for most computing environments, yet it can become a limitation for organisations running highly specialised workloads on carefully tuned hardware platforms. At Google’s scale, where millions of processes and services are executed across global data centres, even relatively minor scheduling inefficiencies can have a noticeable impact on overall performance.

Traditionally, organisations seeking workload-specific scheduling behaviour have had two principal options. The first is to modify the kernel directly, a route that introduces operational complexity and increases the risk associated with deployment. The second is to use user-space scheduling daemons, although these often lack the responsiveness required for microsecond-level scheduling decisions.

Google’s engineers concluded that neither approach offered sufficient flexibility. Instead, the company turned to the Linux kernel’s increasingly important programmable infrastructure, based around eBPF, a technology that allows developers to safely execute custom code within the kernel environment.

Google’s work builds upon earlier initiatives and now increasingly uses the Linux kernel’s scheduler, which allows scheduling policies to be implemented through eBPF programs. The concept is relatively straightforward: rather than embedding all scheduling behaviour into the kernel itself, policy decisions can be loaded dynamically and altered as workload requirements change. This means developers can implement specialised scheduling strategies without needing to patch the base kernel or reboot production systems. The resulting architecture enables scheduling policies to evolve much more rapidly than would otherwise be possible.

Three key optimisation techniques

Google’s custom scheduling framework relies on several complementary mechanisms. The first involves dynamic priority adjustment. Using eBPF maps, information about task state can be shared efficiently between user space and the kernel. This allows scheduling priorities to be modified almost instantly as new RPC requests enter the system and are assigned to worker threads. Because the updates occur directly within the kernel execution environment, they can be performed with very low overhead. This enables workload characteristics to influence scheduling decisions in real time.

The second optimisation focuses on hardware cache locality. Modern server processors contain increasingly complex cache hierarchies. On AMD systems, for example, processing cores are grouped into structures known as Core Complexes (CCX). Memory access patterns often perform better when related tasks remain within the same cache domain. Google’s scheduling policy therefore attempts to pack threads sharing an address space into particular cache regions. The strategy can improve cache utilisation and reduce costly memory accesses across processor boundaries. Importantly, the policy is configurable, allowing engineers to balance memory locality against scheduling latency depending on the workload being executed.

The third technique changes how run queues are managed. Conventional schedulers often maintain run queues on a per-CPU basis. Google’s approach instead manages work across groups of CPUs, providing a coarser level of granularity. According to the company, this improves both workload conservation and fairness across large multi-core systems.

Why eBPF matters

While eBPF is often associated with networking, observability and security applications, its growing role in operating system extensibility may prove equally significant. The technology’s appeal lies in three core characteristics. First, eBPF operates directly within the kernel, providing the low latency necessary for rapid scheduling decisions.

Second, it enables policy and mechanism to be separated. Developers can load, replace and remove scheduling logic without altering kernel source code. This accelerates experimentation and reduces the operational burden associated with infrastructure optimisation. Third, eBPF incorporates a stringent verification process. Before execution, programs are analysed by the kernel’s eBPF verifier to ensure they satisfy defined safety requirements. This helps prevent memory corruption, infinite loops and other classes of failure that could otherwise destabilise production systems. For organisations managing large-scale infrastructure, this combination of performance and safety is highly attractive.

Performance improvements measured in single digits can appear modest in isolation. Yet within hyperscale environments, they often translate into substantial business value. A five percent increase in QPS can allow more requests to be processed using the same hardware estate. Similarly, double-digit throughput improvements for memory-intensive workloads can postpone expensive infrastructure upgrades while improving user experience.

Just as significant is the operational flexibility gained through the ability to deploy new execution policies dynamically. Historically, introducing kernel-level scheduling changes carried significant risk. Testing often required specialised environments, lengthy validation cycles and carefully managed maintenance windows. By contrast, Google’s eBPF-based approach allows engineers to experiment with production scheduling policies more safely and iterate at a much faster pace.

Google has indicated that CPU scheduling represents only the beginning of its broader eBPF strategy. The company is exploring the use of eBPF-driven memory management policies, extending the same principle of separating policy from mechanism into another critical subsystem. One area under investigation involves out-of-memory (OOM) management. Google is assessing an in-kernel framework that would allow eBPF programs to trigger OOM actions proactively and apply alternative victim-selection logic.

The company is also evaluating approaches for more flexible page-cache management. Techniques such as the proposed cache_ext framework could allow applications to influence cache eviction behaviour dynamically, helping systems deliver better performance from existing hardware resources.

Google’s work illustrates a broader shift taking place within cloud computing and operating systems engineering. Rather than relying on increasingly complex monolithic kernels, organisations are moving towards programmable infrastructure where execution policies can be tailored to individual workloads. As data centres continue to face growing demands from artificial intelligence, real-time analytics and large-scale distributed applications, the ability to customise low-level operating system behaviour without sacrificing stability could become a significant competitive advantage.

For now, Google’s results demonstrate that clever scheduling still matters. Even in an era dominated by AI accelerators and advanced processors, deciding which task runs where, and when, remains one of the most powerful optimisation tools available to modern computing infrastructure.