Kubernetes 1.34 makes in-place pod resizing stable after four years

You can finally change CPU and memory requests on a running pod without a restart. The feature took four years and three API revisions to reach general availability.

Younes Bekrar11 min read
ShareXLinkedInFacebook
Kubernetes 1.34 makes in-place pod resizing stable after four years

Kubernetes 1.34 landed on Monday with in-place pod vertical scaling promoted to stable, closing a KEP that opened in 2022 and a feature request that predates it. Until now, changing the CPU or memory request on a running pod required recreating it, which meant a restart, which meant either downtime or enough replicas to absorb the churn. The new behavior lets a controller patch the resources field on a running container and have the kubelet apply it through cgroup updates, with no process restart for CPU changes and an optional restart policy per resource for memory. The release also graduates dynamic resource allocation for GPUs and ships a rewritten scheduler queue.

Why this took four years

The concept is simple and the implementation touches everything. Resource requests feed the scheduler's placement decisions, so changing them after placement means a pod might no longer fit on the node it occupies. The original design punted by allowing only increases that fit within existing node capacity, then had to handle the case where two pods resize simultaneously and race for the same headroom. Memory decreases are worse: shrinking a cgroup limit below current usage triggers the out of memory killer, which is not a graceful outcome.

example.ts
typescript
export async function handler(request: Request): Promise<Response> {  const started = Date.now();  const upstream = await fetch(request);  const headers = new Headers(upstream.headers);  headers.set("x-skarvonix-ms", String(Date.now() - started));  return new Response(upstream.body, {    status: upstream.status,    headers,  });}

The resolution splits behavior by resource. CPU changes apply immediately in both directions because the CFS scheduler tolerates it. Memory increases apply immediately. Memory decreases are governed by a per-container resizePolicy that defaults to RestartContainer, so the naive case does the safe thing. Teams that know their workload can shrink safely can set NotRequired and take responsibility. That per-resource policy field was the last major API change, added in 1.31 after feedback from Google and Datadog on production behavior.

What this unlocks for autoscaling

The Vertical Pod Autoscaler has existed for years and has been effectively unusable in production for stateful workloads because its only enforcement mechanism was eviction. Recommending a better memory request meant killing the pod to apply it, which is unacceptable for a database or a stateful stream processor. VPA 2.0, released alongside Kubernetes 1.34, uses in-place resize as its default actuation path and falls back to eviction only when the node cannot accommodate the change.

The practical effect on cost is significant for anyone who has overprovisioned out of caution. Datadog's platform team, which ran the feature in alpha across an internal cluster of roughly 40,000 pods, reported reclaiming 22 percent of requested memory and 31 percent of requested CPU over six weeks, because requests could be tightened continuously rather than set once at a fearful upper bound. Those are internal numbers from a team with unusual instrumentation, and typical results will be smaller, but the direction is not in doubt.

The story is rarely the launch. It is what breaks, what ships, and who owns the mess at 2 a.m.
Younes Bekrar

The other headline features

Dynamic resource allocation reached general availability, which matters more than in-place resize for anyone running accelerators. DRA replaces the device plugin model with a scheduler-aware allocation system that understands GPU topology, memory partitions, and multi-instance configurations. Nvidia's DRA driver shipped a 1.0 the same day, and it finally allows a pod to request something like eight gigabytes of a partitioned H200 rather than an entire card. AMD and Intel drivers are at 0.9 and expected to follow within a quarter.

The scheduler itself got a rewritten queueing implementation that the SIG Scheduling team says reduces tail scheduling latency by 40 percent in clusters above 5,000 nodes. The change is internal and requires no configuration. There is also a set of smaller quality items: kubectl gained a diff mode that renders server-side apply conflicts readably, and the API server now rejects unknown fields in custom resources by default, which will break some controllers written carelessly.

Upgrade considerations

In-place resize is stable but the ecosystem around it is not uniformly ready. Any admission webhook that validates pod specifications needs to handle the resize subresource, and several popular policy engines shipped support only in the last month. Kyverno covered it in 1.15, OPA Gatekeeper in 3.21. Teams running older versions will find resize requests either rejected or silently bypassing policy, and the second failure mode is worse than the first.

Monitoring also needs attention. A pod whose resources change over time breaks dashboards that assume requests are static, and cost allocation tools that snapshot requests hourly will produce nonsense for workloads that resize frequently. Kubecost and OpenCost both have handling in progress. The safest rollout is to enable the feature, leave VPA in recommendation-only mode for a few weeks, and confirm your cost and capacity tooling reports sensible numbers before letting anything actuate automatically.

The longer arc

Kubernetes at version 1.34 is a mature system adding features that make it cheaper to operate rather than features that expand what it can do. That is the correct phase for infrastructure this widely deployed, and the release notes read accordingly: fewer new objects, more attention to the behavior of existing ones under load. The project's release cadence dropped to three per year in 2025, which most operators welcomed.

The open question is whether resource management moves further toward automation. With in-place resize and DRA both stable, the ingredients exist for a cluster that continuously tunes itself without human capacity planning. Several vendors, including Cast AI and Zesty, already sell that as a product. Whether the upstream project absorbs that capability or leaves it to commercial layers is a governance question SIG Autoscaling will argue about at the next contributor summit in Amsterdam.


Skarvonix will keep following this beat with reporting grounded in how systems behave outside the launch keynote.

  • Kubernetes

Keep reading