GKE Utilization Remains Low: Rightsizing, Spot VMs and GPU Sharing

Google’s long experience running Kubernetes helped shape Google Kubernetes Engine (GKE), including Autopilot’s automation of infrastructure management. Yet operating a managed Kubernetes platform doesn’t automatically make its workloads efficient. The 2026 State of Kubernetes Optimization Report found low average utilization on GKE, pointing to a persistent gap between managing cluster infrastructure and knowing what an application actually needs.

The report measured average CPU utilization of 8%, memory utilization of 20% and GPU utilization of 6%. It also found CPU overprovisioning in 69% of GKE clusters and memory overprovisioning in 79%. Those figures come from direct measurements of more than 23,000 Kubernetes clusters and millions of compute resources across AWS, Google Cloud and Azure, collected before automated optimization was enabled.

The 2026 State of Kubernetes Optimization Report describes an infrastructure platform that can provision and schedule resources, but can’t infer whether an application’s declared CPU and memory needs reflect its actual behavior. That judgment still requires workload data and informed operational choices.

Start with workload requests

Low utilization can look like a node capacity problem, but reducing or expanding a node pool won’t correct a workload request that far exceeds normal use. A service asking for two CPUs and 2 GiB of memory when it typically uses about 500 millicores and 700 MiB is signaling Kubernetes to reserve more than it usually consumes.

The consequences depend on the GKE operating model. In Autopilot, an oversized request directly affects the cost of running the workload. In Standard, requests also shape scheduling: when declared needs exceed actual use, the cluster may require more node capacity than the application consumes. Rightsizing can therefore reduce the requested footprint in either mode, and may also reduce the nodes required in Standard.

Requests should include enough room for real production variation. The waste comes when a cautious estimate becomes a permanent setting without anyone checking whether the original conditions still apply. Compare declared requests with observed CPU and memory use across a representative period. For workloads with weekly traffic patterns, the report suggests looking at seven to 14 days rather than relying on a quiet-hour snapshot.

Begin with a small group of services, examine their behavior and adjust requests incrementally. If production results match expectations, expand the changes. This gives teams a way to verify the effect before applying new settings across hundreds of deployments.

Automation needs similar care. Vertical Pod Autoscaler (VPA) can change resource requests over time, while the Horizontal Pod Autoscaler (HPA) may use CPU or memory utilization to adjust replica counts. Running both together can create a feedback loop: VPA changes requests, that shifts the HPA calculation, and the resulting change in pods affects utilization again. Recommendation-only VPA mode is one way to start; when both autoscalers are needed, application signals such as request rate or queue depth can give HPA a separate basis for scaling.

The report describes automated rightsizing that cut provisioned CPU by roughly half while reducing out-of-memory kills to near zero. That result doesn’t mean every workload should receive smaller requests. It shows why production measurements can help distinguish useful headroom from excess capacity.

Autoscaling works best with accurate requests

On GKE Standard, workload rightsizing is only part of the task. Teams also need to make effective use of the Compute Engine nodes underneath their applications. Cluster Autoscaler adjusts node capacity, while Node Auto Provisioning can create node pools when a workload needs a machine type that the cluster doesn’t already offer.

Neither mechanism can tell whether a workload’s request is inflated. If services ask for more CPU than they typically use, the autoscaler will respond to those requests and provision enough capacity to meet them. Once requests better match observed needs, scaling and node consolidation have a more useful starting point: workloads may fit more efficiently, appropriate machine types become practical, and unneeded nodes can be removed.

That flexibility gives Standard users more control over node choices, along with responsibility for the capacity they manage. With Autopilot, Google handles more of the infrastructure decisions; in Standard, teams must pay closer attention to how workloads fit the fleet.

Match capacity type to workload behavior

After requests and node placement are in better shape, consider whether every workload needs on-demand capacity. GKE Spot VMs can reduce costs for applications that tolerate interruption. Google’s Spot model provides roughly 30 seconds of preemption notice, so interruption handling should be part of the design.

Batch processing, CI/CD runners, sufficiently replicated stateless services and machine learning jobs that checkpoint their progress are potential candidates. Teams need to account for disruption and termination behavior, and keep an on-demand fallback for times when Spot capacity is unavailable. Otherwise, a lower-cost compute choice can create an availability issue.

A mixed capacity strategy lets workloads with predictable availability needs use dependable capacity while interruption-tolerant jobs use Spot when it’s available. Spot is most useful after teams understand the baseline those jobs actually require.

Put expensive hardware to work

GPU utilization presents a different challenge from CPU and memory requests. The report found average GKE GPU utilization of 6%, with substantial variation through the day, while GPU sharing remained relatively uncommon. When an inference workload uses only part of a GPU most of the time, assigning it an entire device may leave costly capacity idle. Depending on the hardware and application, sharing or partitioning can allow multiple workloads to use the same physical GPU more effectively.

One production example in the report combined GPU time-slicing for seven models with improved bin-packing and Spot use. The change reduced costs by more than 70% compared with the previous platform while maintaining latency. It’s an example of what can be gained by making existing hardware do more useful work before buying additional capacity.

CPU architecture is another choice. The report says ARM accounts for about 9% of the Kubernetes CPU fleet and has grown 3.5 times faster than x86 since the second quarter of 2024. GKE offers ARM options including T2A and Google’s Axion-based C4A and N4A families. Teams with workloads that can run on linux/arm64 can move a small set first, then compare cost and performance before widening the change. A cheaper processor won’t fix an oversized workload request by itself.

Commit after establishing the baseline

Pricing commitments make more sense once workload and infrastructure needs are understood. Google Cloud’s Sustained Use Discounts apply automatically to eligible Standard node usage. Committed Use Discounts involve a deliberate commitment against a baseline. If teams commit before identifying that baseline, today’s excess capacity can become a longer-term cost.

The practical sequence is to align requests with production behavior, improve node placement, use Spot for workloads that tolerate interruptions, and assess GPU sharing or ARM for suitable applications. Then set commitments around the capacity workloads consistently need, while keeping burst capacity distinct.

GKE automates many difficult infrastructure tasks, but utilization still depends on choices made around individual workloads and their hardware. A controlled rollout, measured in production before it expands, gives teams a concrete way to improve that utilization while keeping service behavior in view.