From Karpenter to GKE ComputeClasses, Part 3: accelerators, reservations, and the portability question

Part 3 of a 3-part series on porting EKS/Karpenter autoscaling patterns to GKE. Part 1 covered the mental model — a solver searching a space versus an author ranking a list. Part 2 covered diversity, consolidation, and the where/when split between ComputeClass and WorkloadClass. This one is the expensive end of the fleet, plus the promised answer on portability.

Everything in the first two posts was, in a sense, low-stakes. Get a CPU priority list slightly wrong and you overpay by some percentage on nodes that cost a few cents an hour. Get an accelerator priority list wrong and you’re either paying for an idle H200 or explaining to someone why a seven-hour training run restarted from a checkpoint at hour six.

So this post is about the parts of the model where the defaults stop being safe: the capacity ladder for accelerators, reservations (the single field most likely to silently eat your fallback), the toleration trap that’s inverted from the one you know on EKS, and how a ComputeClass gets fenced off per tenant. Then I close the loop on the question I’ve deferred twice: what happens to Karpenter-vs-ComputeClass parity as GKE open-sources its autoscaling stack.

Karpenter picks your capacity order for you. For GPUs, it picks wrong.

#

Start with the thing that surprises people most, because it’s a default rather than a setting and defaults are invisible until they hurt.

Karpenter has a capacity-type hierarchy, and it is not the order you write in your YAML. Karpenter provisions capacity reservations first, then Spot, then On-Demand — cheapest-obtainable-first, essentially — and the sequence you list capacity-type values in conveys nothing. That’s a sensible default. For a CPU fleet it’s very close to what you’d have written anyway: burn the capacity you already paid for, then take the cheap interruptible stuff, then pay list price.

For accelerators it’s backwards, and the reason is that the cost of an interruption changes completely once the workload is a GPU one:

• Inference: a Spot preemption mid-serving drops in-flight requests and takes minutes to recover, because accelerator nodes are slow to start — image pull, driver install, model weights load. The node is cheaper; the p99 is not.
• Training: a preemption mid-step forces a restart from the last checkpoint. You don’t lose the node, you lose the hours since your last checkpoint — on the most expensive hardware you rent.

What you actually want for accelerators is reservation → On-Demand → queued → Spot: pay first from capacity you’ve already committed, then pay list price for uninterruptible capacity, then wait in a queue if you must, and only take interruptible capacity as a genuine last resort. Notice that On-Demand sits above Spot there. That inverts Karpenter’s built-in ordering, and it’s not something you express by reordering a list in a Karpenter NodePool — you express it by splitting into multiple NodePools and hanging weight off them.

In a ComputeClass, ordering is the whole API. There is no built-in hierarchy competing with yours:

apiVersion: cloud.google.com/v1
kind: ComputeClass
metadata:
  name: genai-inference-g4
spec:
  nodePoolAutoCreation:
    enabled: true
  # NOTE: no priorityDefaults.location — it would collide with the Specific
  # reservation on priority 1 (see the next section).
  priorities:
  # 1. Specific reservation — pre-paid, no queue, no preemption.
  - gpu:
      count: 1
      type: nvidia-rtx-pro-6000
      driverVersion: default
    machineType: g4-standard-48
    spot: false
    reservations:
      affinity: Specific
      specific:
      - name: g4-inference-reservation
        zones: ['us-central1-a']   # reservations are zonal; scope lives here

  # 2. On-Demand floor — the latency-safe fallback, deliberately above Spot.
  - gpu:
      count: 1
      type: nvidia-rtx-pro-6000
      driverVersion: default
    machineType: g4-standard-48
    spot: false
    location:
      zones: ['us-central1-a', 'us-central1-b', 'us-central1-c']

  # 3. DWS FlexStart — accept a queue (~3 min) rather than take Spot.
  - flexStart:
      enabled: true
    capacityCheckWaitTimeSeconds: 1800
    gpu:
      count: 1
      type: nvidia-rtx-pro-6000
      driverVersion: default
    machineType: g4-standard-48
    location:
      zones: ['us-central1-a', 'us-central1-b', 'us-central1-c']

  # 4. Spot — absolute last resort; only safe if replicas mask a preemption.
  - gpu:
      count: 1
      type: nvidia-rtx-pro-6000
      driverVersion: default
    machineType: g4-standard-48
    spot: true
    location:
      zones: ['us-central1-a', 'us-central1-b', 'us-central1-c']

Three notes on that, because each one is a place people go wrong:

• Priority 3 is the rung with no AWS analog. DWS FlexStart is queued provisioning: instead of failing when accelerators are unavailable right now, you take a ticket and GKE hands you the capacity when it frees up (typically a few minutes). It’s a fundamentally different answer to a stockout than “fall back to something worse” — for batch and training, waiting three minutes usually beats landing on Spot. One caveat worth keeping straight: maxRunDurationSeconds on a FlexStart node caps how long it runs, it does not improve your odds of getting one.
• machineType rather than machineFamily here is deliberate. Part 1 pushed intent-based priorities hard, and for CPU that’s right. Accelerators are the exception: the GPU model, the host shape, and the attached NVLink/local-SSD topology come as a matched set, so pinning the exact shape is the honest expression of what you need.
• Part 1’s warning about Spot inverts here. On GCP, Spot and On-Demand CPU frequently share the same underlying pool, so a Spot stockout usually means an On-Demand stockout too — hence “always end with an On-Demand floor.” Accelerators don’t behave that way. Spot GPU capacity is often available when On-Demand isn’t, which makes a bottom-rung Spot tier genuinely useful rather than decorative.

And a callback to Part 2: autoscalingPolicy.gpuConsolidationThreshold is the GPU-scoped twin of consolidationThreshold. Set it separately and set it conservatively. A half-idle GPU node is expensive, but reclaiming one out from under a model that’s still warming up is worse.

The reservation field that quietly eats your fallback

#

Reservations are where the two systems diverge structurally, not just cosmetically.

On the Karpenter side, On-Demand Capacity Reservations arrived as a first-class reserved capacity type in v1.3, and you select them with capacityReservationSelectorTerms on the EC2NodeClass — by tag, usually, ANDed together. The selection lives on a different object from the NodePool that ranks capacity, and it’s tag-driven, so “which reservation will this actually draw from” is answered by whatever happens to be tagged in the account right now.

On GKE, the reservations block sits inside a single priorities[] entry, so the reservation is part of the same ranked rung as the machine shape and the Spot flag. One object, one list, one place to look. What you give up is tag-based dynamism; what you get is that the rung and its capacity source can’t drift apart.

The field to get right is affinity, and one of its values is a trap:

AnyBestEffort is the one that will cost you money quietly. It resolves the “no reservation capacity” case at the GCE layer, by buying On-Demand — which means the priority succeeds, and the autoscaler never walks down to the rungs you wrote underneath it. Your carefully-ranked FlexStart and Spot tiers are unreachable, not because they’re broken, but because nothing ever fails hard enough to reach them.

If you take one line from this section: a fallback ladder only works if the rungs above it are allowed to fail.

Two schema details that bite immediately after:

• location.zones cannot be combined with affinity: Specific. You’ll get “location config with specific reservations enabled.” Reservations are zonal, so the zone scope comes from reservations.specific[].zones instead. A policy-only location.locationPolicy: BALANCED is still fine. This is also why the G4 example above has no priorityDefaults.location — a default that’s illegal on priority 1 breaks the whole class.
• One priority naming several zonal reservations, not one priority per zone. Per-zone priorities are traversed sequentially, so you’ll fully drain zone a before touching zone b. List all the zonal reservations inside a single rung with locationPolicy: BALANCED and you get even node spread across them.

While we’re here — the reservation/CUD distinction trips up nearly everyone porting a committed-spend posture from AWS. Committed Use Discounts apply automatically: if you hold an n4 CUD and the autoscaler provisions an n4, the discount attaches with no ComputeClass configuration at all. Reservations do not. They’re capacity you’ve set aside, and they’re consumed only when something explicitly targets them. Aligning your machineFamily choices with your CUDs is a billing optimization you get for free; wiring up reservations is work you have to actually do.

The toleration trap, inverted

#

Part 1’s number-one day-one failure was untranslated Pod node selectors. Here’s its accelerator sequel, and it catches people precisely because they did the right thing on EKS.

On EKS, GPU nodes are not automatically tainted. If you want nvidia.com/gpu:NoSchedule on your GPU nodes so that general workloads don’t squat on them, you add it yourself in the Karpenter NodePool’s spec.template.spec.taints. Plenty of teams don’t bother — if a NodePool only ever produces GPU instances and only GPU pods select it, the taint is redundant. Those teams have GPU pod specs carrying no toleration at all, and that’s been fine for years.

On GKE, the taint is applied for you. GKE auto-taints GPU nodes with nvidia.com/gpu:NoSchedule, unconditionally.

So the pod spec that worked on EKS because nobody tainted anything arrives on GKE and sits Pending with a noScaleUp event. Nothing in the ComputeClass is wrong. The autoscaler is behaving correctly — it’s declining to scale up for a pod that can’t tolerate the node it would create.

The fix is two lines on the PodSpec:

tolerations:
- key: nvidia.com/gpu
  operator: Exists

The obvious next question is whether the Spot rung of the ladder above behaves the same way, since GKE taints Spot nodes with cloud.google.com/gke-spot=true:NoSchedule. That one I’d state more carefully than I first wanted to. On Autopilot it’s explicitly handled for you — Spot Pods get both the taint and the matching toleration. Off Autopilot, the documentation that tells you to add the toleration yourself is describing the older cluster-level node auto-provisioning path, not pools auto-created by a ComputeClass; and the ComputeClass troubleshooting guide, which does name a missing GPU toleration as a cause of Pending, never mentions Spot at all. My own experience running these is that I’ve not had to add a Spot toleration to a pod spec consuming ComputeClass capacity. Treat it as cluster-verify-before-you-assume rather than a rule in either direction.

The GPU taint is the one I’d state flatly. And it’s worth being precise about why it catches people: ComputeClasses do auto-tolerate their own cloud.google.com/compute-class taint on auto-created pools, so it’s a reasonable inference that tolerations are handled across the board. That inference is what breaks. If you write one debugging note to yourself from this whole series, make it that a Pending accelerator pod is a toleration problem until proven otherwise.

TPUs: the rung with no translation table

#

Part 1 gave you an AWS-to-GCP family translation table. TPUs are where that table runs out — there’s no m6i-to-n4 equivalence to make, because there’s no AWS analog to map from. So this is less a porting exercise than a new shape to learn, and it’s short:

priorities:
# 1. Specific reservation — pre-paid TPU capacity, no queue, no preemption.
- tpu:
    type: tpu-v5-lite-podslice
    count: 8
    topology: 2x4
  spot: false
  reservations:
    affinity: Specific
    specific:
    - name: tpu-v5e-training-reservation
      zones: ['us-central1-a']

# 2. On-Demand — guarantees forward progress when the reservation is exhausted.
- tpu:
    type: tpu-v5-lite-podslice
    count: 8
    topology: 2x4
  spot: false
  location:
    zones: ['us-central1-a']

# 3. Spot — cheapest, and only safe with frequent checkpointing.
- tpu:
    type: tpu-v5-lite-podslice
    count: 8
    topology: 2x4
  spot: true
  location:
    zones: ['us-central1-a']

The thing to internalize is that topology is not a sizing hint — it’s the physical interconnect shape of the slice, and a workload compiled for 2x4 wants 2x4. That’s why every rung here repeats the same accelerator block: the fallback is over capacity source, never over topology. And it’s single-zone throughout, because TPU reservations are zonal and a slice can’t straddle zones.

Same ladder philosophy as the GPU class, same reason Spot sits at the bottom.

Kueue: the layer that’s already portable

#

For batch and training, the honest answer to “what’s my Karpenter here” is partly “the same thing you already run on EKS.” Kueue is a vendor-neutral CNCF project handling job queuing, quota, and admission, and the seam between it and cloud-specific provisioning is clean on both clouds:

Kueue decides whether a job is admitted. The provisioner decides what hardware it lands on.

The join is a node label. A Kueue ResourceFlavor points at a ComputeClass:

apiVersion: kueue.x-k8s.io/v1beta1
kind: ResourceFlavor
metadata:
  name: tpu-v5e-flavor
spec:
  nodeLabels:
    cloud.google.com/compute-class: tpu-v5e-training

When Kueue admits a job under that flavor, it injects the nodeSelector into the pods, and GKE’s autoscaler walks the ladder above. On EKS, the same ResourceFlavor points at karpenter.sh/nodepool instead. Your queues, cohorts, quotas, borrowing rules, and ClusterQueue topology — the part that actually encodes your organization’s policy about who gets the GPUs — are identical on both clouds. One label per flavor is the diff.

If you’d rather see that wired up end to end than in fragments, my colleague Michał Żyliński wrote a practical guide to Kueue and Custom Compute Classes that builds precisely this: a ResourceFlavor bound to a cost-optimized ComputeClass, Spot-first with an On-Demand fallback, from cluster creation through a GPU job actually running.

That’s worth dwelling on for a second, because it reframes the portability question that’s been running under this whole series. The layer where your institutional knowledge lives is already vendor-neutral. The layer underneath it, the one that provisions hardware, was always going to be cloud-specific, because the hardware is.

Tenant isolation: one knob versus three layers

#

Karpenter’s isolation story is node-shaped and mostly conventional: a NodePool per tenant, spec.template.spec.taints to keep other workloads off, spec.weight for ordering, and spec.limits (cpu, memory, nodes) as a hard cap on how much compute that NodePool will ever manage. It works, it’s easy to reason about, and it has one notable gap — nothing stops a pod in another namespace from simply selecting your tenant’s NodePool by label. Isolation is by taint, and a wildcard toleration defeats a taint.

GKE splits this into three independent layers, and the reason to know all three is that each one protects something the others structurally cannot:

1. Who can create or edit the class — RBAC. ComputeClass is a cluster-scoped CRD, so this is a ClusterRole plus ClusterRoleBinding on apiGroups: ["cloud.google.com"], resources: ["computeclasses"]. Grant patch and delete alongside create and update, or a non-creator can still edit a class they didn’t make. Bind to a Google Group rather than individual users.

2. Who can request the class — ValidatingAdmissionPolicy. This is the layer people skip, and the reason it’s necessary is a genuine surprise: RBAC cannot restrict consumption. Referencing a ComputeClass from a Pod isn’t a verb on the ComputeClass object — it’s a field in the Pod spec. RBAC governs objects; admission governs specs. There is also no allowedNamespaces field on the ComputeClass spec (this is the single most-hallucinated field in the API — it does not exist). Consumption control is admission-only, via a native CEL ValidatingAdmissionPolicy, no webhook required.

Two things to get right when you write that policy, both of which are silent leaks:

• Close all three access paths. nodeSelector is the obvious one. But nodeAffinity with a matchExpressions key of cloud.google.com/compute-class reaches the same class, and so does a toleration for the class’s NoSchedule taint — including the wildcard form (operator: Exists with no key), which tolerates every taint in the cluster. A nodeSelector-only policy is the classic bypass.
• Match every workload kind. Policies that constrain only pods and deployments leave statefulsets, daemonsets, replicasets, jobs, and cronjobs wide open. Controllers carry the spec at spec.template.spec; CronJob buries it at spec.jobTemplate.spec.template.spec.

Run it with validationActions: ["Audit"] first to find your existing violators, then add Deny and failurePolicy: Fail.

3. How much it can provision — CapacityQuota. New in GKE 1.36.2+ (autoscaling.x-k8s.io/v1beta1), this caps the physical footprint — CPU, memory, GPUs, node count — that workloads consuming a class can autoscale into. It’s the closest analog to Karpenter’s spec.limits, but it does something limits can’t, because it selects on labels rather than being a property of the pool:

selector:
  matchLabels:
    cloud.google.com/compute-class: tenant-a-class
    cloud.google.com/machine-family: n4      # cap the CUD-covered family only
limits:
  resources:
    cpu: 100

That caps n4 at 100 cores — matching, say, a 100-core CUD — without capping the rest of the class. When demand exceeds the commitment, the autoscaler emits a noScaleUp for n4 (exceeded quota: "CapacityQuota/<NAME>") and spills over into the n4d and c4 rungs further down the same ComputeClass. You’ve expressed “use exactly as much of my discounted capacity as I bought, then keep going on undiscounted capacity” — in two objects, declaratively. With Karpenter’s per-NodePool limits you’d model that as separate NodePools and weights, which works, but the commitment boundary ends up encoded in your pool topology rather than stated as a quota.

The portability question, finally

#

I’ve deferred this twice, so let me answer it properly.

The concern is legitimate and I don’t want to wave it off: Karpenter is open source and CNCF-governed, ComputeClasses are a GKE API, and standardizing your provisioning model on a single vendor’s control-plane API is a real cost. If you’re running both clouds, you’re right to weigh it.

Here’s the state of play as of today, and I’ll separate what has actually happened from what’s intended.

What’s already true: at KubeCon EU 2026 Google announced it would open-source GKE Cluster Autoscaler, and that code is now public — Apache 2.0, at GoogleCloudPlatform/cluster-autoscaler. Not an announcement of intent; a repository you can clone and read this afternoon. The stated goal was “a vendor-neutral platform that the OSS community can benefit from and build on top of,” and separately, WorkloadClass — the when-may-I-be- disrupted half from Part 2 — has been built in the open from day one, deliberately, so that it can work across Kubernetes vendors rather than only on GKE.

What’s intended: the ComputeClass API lands in that same repository. I’m driving that effort, and our current target is mid-2027. I want to be straight about what that number is: it’s a target, not a commitment, and dates like this move for unglamorous reasons. Treat it as directional. The GKE team will have a more detailed open-source roadmap announcement in the near future, and that will be a far better source than a blog post from me — when it lands, believe it over this paragraph.

What I’d tell you not to expect from it. An open-sourced API is not the same thing as a working cross-cloud implementation. Publishing the ComputeClass API and the autoscaler that honors it means the model is inspectable, forkable, and available for other vendors and the community to build against. It doesn’t mean the identical YAML provisions nodes on three clouds the day the repo updates — machineFamily: n4 is a GCP concept no matter who owns the code. The realistic near-term win is that the shape of the API — ranked priorities, capacity ladders, provisioning intent separated from disruption policy — stops being something you can only get from one vendor. Which is precisely the bet Karpenter made, and won.

So: if you were holding off on ComputeClasses because you didn’t want to invest in learning an API you might be stuck inside, the direction of travel is pointed away from that, not toward it. If you were holding off because you want a single YAML that provisions on EKS and GKE alike, that’s a longer road, and I’d rather say so than sell you the roadmap.

Wrapping the series

#

Three posts, and the through-line was one idea: Karpenter hands a solver a space to search, and GKE asks an author to declare a ranking. Everything else followed from that — the tie-break tier instead of a wide instance list, ordering instead of weight, optimizeRulePriority instead of drift, ComputeClass and WorkloadClass instead of one bundled NodePool, and here, a hand-written capacity ladder instead of an inherited price hierarchy.

What I’d hope you take away, if you’re the person actually doing the port:

• Part 1’s trap is still the one that gets you. Untranslated Pod node selectors, and now their accelerator sequel: the GPU toleration GKE expects and EKS didn’t.
• The ladder is the artifact worth spending time on. Not the individual fields — the ordered list of “what do I want, and what am I willing to accept instead.” For CPU that’s cheap to get wrong. For accelerators it’s the whole ballgame.
• Let rungs fail. AnyBestEffort, an over-broad whenUnsatisfiable, a priorityScore tier that always succeeds — anything that prevents a priority from failing cleanly makes everything below it decorative.

The ComputeClass patterns across all three posts come from an experimental GKE ComputeClasses agent skill I’ve been building. If you’d rather hand an agent your Karpenter NodePools and get ComputeClasses back than adapt YAML out of a blog post, that’s the shortcut — and it knows about every gotcha in this series, because most of them got encoded there first.