EKS Utilization Lags as Kubernetes Fleets Grow: Rightsizing, GPUs and Spot

Amazon EKS gives teams a wide range of compute options, from Graviton processors and GPU accelerators to Karpenter and EC2 Spot Instances. Yet that choice hasn’t translated into efficient fleets. The 2026 State of Kubernetes Optimization Report finds that utilization fell across production clusters even as overprovisioning climbed. The gap points to an operational problem: Kubernetes capacity is often set by resource requests and static assumptions, rather than measured application demand.

The report analyzed tens of thousands of clusters and millions of compute resources across AWS, Azure and GCP. In 2025, average CPU utilization dropped from 10% to 8%, while memory utilization fell from 23% to 20%. CPU overprovisioning rose from 40% to 69%, and memory overprovisioning reached 79%. Average GPU utilization was 5%, both across the report and on EKS.

Those measurements were taken in production before automated optimization was activated. They describe how the clusters ran day to day, rather than what they might achieve after tuning. For teams paying for managed Kubernetes and underlying EC2 capacity, the distinction matters: a flexible instance catalog can help select capacity, but it cannot make workloads use resources they don’t need.

Karpenter can fit nodes to requests, not consumption

Karpenter was designed to provision nodes in response to pending pods, without requiring teams to maintain rigid, preconfigured node groups. That makes it a more flexible way to respond to scheduling demand than relying on fixed instance group sizes. But its decisions still depend on the resource requests Kubernetes sees. An autoscaler can match nodes to those requests; it can’t determine whether the application’s requests reflect actual usage.

Consider a service that requests 4 vCPUs and 8 GiB of memory but uses 500 millicores of CPU and 1.5 GiB of memory at peak traffic. The scheduler still needs capacity for the requested footprint, so Karpenter may provision a node large enough to accommodate it. The article attributes high CPU overprovisioning in part to this mismatch.

Teams often pad requests because too little CPU can lead to throttling and higher latency, while memory pressure can trigger an out-of-memory termination. Since extra headroom rarely causes an immediate outage, those conservative settings can become lasting defaults in Helm charts and Terraform templates. Across a large fleet, small buffers multiply into substantial unused capacity. With self-managed EC2 nodes on EKS, the organization pays for that space.

GPU sharing changes the cost equation

Underused GPUs have a steeper cost than idle CPU cores. The source reports that AWS raised H200 Capacity Block prices by 15% in January 2026, citing supply and demand, while EKS GPU utilization remained at 5%. Inference workloads can be especially uneven: an endpoint may receive bursts of requests and then sit idle, while its dedicated GPU instance continues to incur hourly charges.

Sharing a physical GPU across workloads through time-slicing, alongside automated bin-packing, can better match capacity to those changing loads. The report describes edtech platform ALLEN Digital, which ran seven machine learning models—including BGE-M3, LlamaGuard and Multilingual E5 Large. The models had previously used dedicated Amazon SageMaker instances. After moving to EKS and using GPU time-slicing, automated bin-packing and a 50/50 mix of Spot and on-demand capacity, the company consolidated models on shared GPUs. The source says compute costs fell by more than 70%, with no change to inference latency.

That result illustrates the potential of sharing, but it comes from one reported deployment. Teams considering a similar design will need to check whether their models’ traffic patterns, performance requirements and workload isolation needs suit shared accelerators.

Spot savings depend on where capacity runs

Spot Instances can cost up to 90% less than on-demand capacity, yet fewer than 2% of GPUs in the report ran on Spot. The concern is understandable: Spot capacity can be interrupted, with two minutes’ notice. But the source’s data suggests the risk varies substantially by region.

For NVIDIA T4 instances of type g4dn.xlarge, the reported 24-hour survival probability remained above 90% in eu-west-3. In us-east-1 and eu-central-1, it fell below 20% over the same period. Price variation also matters: Spot pricing for p4d.24xlarge A100 instances differed by two to nearly five times across the cited US regions. The report says shifting workloads in response to market conditions produced savings of up to 80% compared with the average Spot price.

Following regional availability, interruption patterns and prices manually is a demanding task for an engineering team. Automated rebalancing can handle those changes alongside workload placement. Marketing platform Iterable is cited as using hourly Spot rebalancing across its AWS footprint. The report attributes to that work the removal of 2,000 unneeded CPUs, a 10% reduction in node count and a 75% decrease in compute spending while serving variable real-time traffic.

Graviton helps when workloads are ready to move

ARM CPU nodes grew 3.5 times faster than x86 nodes from the second quarter of 2024, reaching 9% of the Kubernetes CPU fleet in the report. It also found that 93% of observed deployments were stateless. Stateless services, workers and APIs are often candidates for Linux on ARM because they don’t depend on persistent volumes or storage I/O in the same way that stateful applications can.

Moving an oversized workload to Graviton can lower per-core costs, but changing processor architecture alone leaves the oversized request in place. Rightsizing first, then packing the workload onto Graviton capacity, can address both the excess allocation and the hardware cost. Compatibility still needs to be checked for application images and dependencies before a move.

Static headroom doesn’t guarantee reliability

Large resource buffers are often justified as protection for service-level objectives. The report challenges that assumption with one production environment where out-of-memory kills averaged 40 to 50 per measurement window and sometimes exceeded 80, despite extensive resource padding.

Broad, static buffers can leave quiet services with excess allocations while a fast-growing or memory-leaking workload continues using limits that no longer match its needs. Automated rightsizing can reduce CPU and memory requests when measured usage is consistently lower, then increase memory limits when pressure appears. In the cited environment, this approach brought out-of-memory kills close to zero while cutting provisioned CPU by about 50%.

Commit after removing the waste

Savings Plans and Compute Reserved Instances can lower costs, but commitments made against an oversized baseline can lock in unneeded capacity. The source estimates that manually managed workloads and traffic shifts can leave up to 35% of prepaid commitments unused. Its recommended sequence is to rightsize requests against production usage, pack nodes efficiently, direct interruption-tolerant workloads to Spot with multi-zone options and on-demand fallback, then commit to the stable baseline that remains. It says this can keep commitment utilization near 98%.

The EKS optimization guide provides further detail. The report’s findings across EKS, AKS and GKE frame low utilization as a Kubernetes operations challenge across cloud providers. As clusters spread across teams, regions and instance types, requests, placement and node capacity need regular attention. The practical starting point is to measure real demand, then make capacity decisions that can adapt as that demand changes.