From Karpenter to GKE ComputeClasses, Part 2: diversity without a solver

Part 2 of a 3-part series on porting EKS/Karpenter autoscaling patterns to GKE. Part 1 covered the core mental model — a solver searching a space versus an author ranking a list. Part 3 covers accelerators, reservations, and tenant isolation.

Part 1 left one thread hanging on purpose. A Karpenter NodePool solver quietly does two jobs at once: it picks among a wide set of interchangeable instance types — across families and architectures alike — and it enforces how aggressively nodes get disrupted while doing so. Collapse that into a ranked priorities[] list and the ordering answers “what do I want first” — but you still need explicit answers for “what if several things are equally fine” and “how much disruption am I willing to eat while GKE optimizes underneath me.” This post is those answers, plus the one where GKE and Karpenter diverge hardest: disruption isn’t controlled by the same object that provisions the node anymore.

Same preference, several acceptable answers

#

A ranked list implies a strict order — try this, then that, then the next thing. But plenty of real preferences aren’t strict: “any of these three families, whichever is cheapest and available right now” is a single preference with several acceptable answers, and encoding it as three separate priority tiers would just go stale the moment prices or capacity shift.

priorityScore is the escape hatch. Priorities that share a score form one tier and are evaluated together — GKE resolves the tie by lowest unit cost among whichever members of the tier currently have capacity:

priorities:
# Tier 1: three interchangeable Gen-4 families, cost tie-break decides.
- machineFamily: c4d
  priorityScore: 100
- machineFamily: c4
  priorityScore: 100
- machineFamily: n4
  priorityScore: 100
# Tier 2: broader-availability fallback.
- machineFamily: n2d
  priorityScore: 50
- machineFamily: n2
  priorityScore: 50
# Floor: widest zone footprint, guarantees execution.
- machineFamily: e2
  priorityScore: 10

Three rules worth internalizing, because they’re stricter than they look:

• A tier caps out at 3 rules — for now. Need more equivalent options than that? They spill into a lower-scored tier — you can’t flatten five families into one preference today. priorityScore is a newer field, and this cap is a temporary constraint of its early rollout, not a permanent ceiling; expect it to loosen as the feature matures.
• It’s all-or-nothing. The moment one priority in the class sets priorityScore, every priority must. There’s no mixing strict-order rules with scored ones.
• Requires GKE 1.35.2-gke.1842000 or later. Without a score, GKE falls back to plain declaration order — which is also your fallback if you’re on an older cluster.

Without priorityScore, you’d have to pick one family and hope, or hand-rank three functionally-identical options in an order that’s really just a guess about which one has capacity this week. The tier is what actually replaces “let the solver pick” — scoped to the one decision where picking pays off, not applied to the whole class.

Diversity across architecture, not just family

#

The other axis Karpenter’s solver hides is architecture — mixing Arm and x86 in one requirements block and trusting the scheduler to land pods correctly. GKE does this too, but two things have to be true simultaneously or pods stick on the fallback tier forever, and neither failure looks like an architecture problem when you’re staring at a Pending pod.

priorities:
- machineFamily: c4a       # Axion (Arm64) — price/performance for scale-out, stateless work
  spot: false
- machineFamily: c4        # x86 fallback when Arm capacity is short
  spot: false
activeMigration:
  optimizeRulePriority: true   # pull pods back onto c4a once Arm capacity returns

• The image has to be multi-arch. A pod scheduled onto a c4a node needs an arm64-compatible manifest. An amd64-only image will land on the node and then fail to run — not fail to schedule, which makes it a worse debugging experience than a Pending pod. If your image isn’t multi-arch, don’t reach for this pattern; pin kubernetes.io/arch: amd64 in a nodeAffinity instead and skip Arm entirely.
• Arm nodes carry a taint (kubernetes.io/arch=arm64:NoSchedule). Your pod spec needs a matching toleration or it sits on the x86 priority even when Arm capacity is wide open, and again — nothing in the ComputeClass looks wrong.

On Autopilot, there’s a third option that skips both gotchas: set podFamily: general-purpose-arm on a priority and GKE handles the multi-arch and toleration mechanics for you. Worth knowing about even if Standard is where you live today, since it’s the direction auto-creation is generally headed — less for you to wire up by hand, not more.

Consolidation and drift, one degree deeper

#

Part 1 introduced consolidationDelayMinutes — how long an underused node waits before reclaim, floor one minute. There are two more knobs under autoscalingPolicy worth knowing once you’re tuning for real:

• consolidationThreshold — the CPU/memory utilization percentage below which a node becomes eligible for removal at all. consolidationDelayMinutes is a timer; consolidationThreshold is the gate the timer doesn’t even start counting toward until utilization drops under it. Tune them together: a low threshold with a short delay bin-packs aggressively; a high threshold with a long delay favors stability.
• gpuConsolidationThreshold — the same idea, scoped to GPU utilization. More relevant once you’re running accelerators, which is Part 3’s territory.

And activeMigration.optimizeRulePriority — used twice already in this post — is the answer to Karpenter’s drift, but it goes one better. Karpenter’s drift is purely reactive: it only acts when you change the NodePool spec, at which point existing nodes are flagged out-of-spec and replaced. It doesn’t sit there hunting for something better than what it already has. optimizeRulePriority does — GKE continues watching after initial placement, and when a higher-priority rule in your list becomes satisfiable (capacity frees up, a Spot price drops back down), it provisions a replacement node against that rule and migrates the workload up onto it, with no spec change from you required to trigger it. Off by default, because migrating a running pod is itself a disruption. The principle underneath is worth stating explicitly: a ComputeClass is willing to land fast on whatever the priority list currently allows, then consolidate upward toward your highest priorities over time — which is exactly the segue into the part of this post that’s actually new.

Disruption control lives on the workload, not the ComputeClass

#

Here’s the deliberate seam from Part 1’s concept table, expanded: Karpenter bundles where a pod runs and when it may be disrupted into the same NodePool, via disruption.budgets — an array of node-percentage caps, optionally scoped to a cron schedule and duration, optionally restricted to specific reasons (Empty, Drifted, Underutilized):

spec:
  disruption:
    budgets:
    - nodes: "20%"
      reasons: ["Empty", "Drifted"]
    - nodes: "0"
      schedule: "@daily"
      duration: 10m
      reasons: ["Underutilized"]

Read that carefully and notice what it’s actually expressing: a rate limit on the node pool — no more than 20% of nodes disrupted at once, and none at all during a 10-minute daily window. That’s a real, useful control. It is not, however, a way to say “this specific workload needs two full days of guaranteed uptime before it’s touched” or “only ever disrupt these pods on a Saturday-night maintenance window.” A budget governs the pool; it has no idea which workload is sitting on the node it just reclaimed.

GKE splits this deliberately, and I called it “the interesting one” in Part 1 for a reason: a ComputeClass decides where a workload runs, full stop. Disruption tolerance is a property of the workload, so it belongs on the workload — which is what WorkloadClass is for.

I want to be straight about what this is today, because it’s not folded into the GKE control plane the way ComputeClasses are — it’s a separate controller you install yourself (kubectl or Helm, with cert-manager as a prerequisite). It’s being built by GKE product and engineering, and it lives in gke-labs, but don’t read that placement as “not mature yet.” It’s deliberate: the lesson learned from ComputeClass is that tying an API to one vendor’s control plane from day one narrows who can adopt it, so WorkloadClass is being built OSS from the start, specifically so it can work across Kubernetes vendors, not just GKE. It’ll likely land in core GKE too, in time — but the labs home isn’t a waiting room, it’s the point. And it’s a preview of where ComputeClass itself is headed, per the open-sourcing direction I flagged in Part 1: the same portability WorkloadClass is designed for from day one is what ComputeClass is growing into. I’ll have more to say about where both are headed in Part 3.

What it does today, concretely: two CRDs, cluster-scoped guardrails and namespace-scoped policy.

apiVersion: workloads.gke.io/v1
kind: WorkloadClassGuardrail
metadata:
  name: default
spec:
  constraints:
    disruption:
      allowedDisruptionDays: [Saturday, Sunday]
      maxAllowedWindows: 2
      maxNonDisruptionDurationDays: 30
---
apiVersion: workloads.gke.io/v1
kind: WorkloadClass
metadata:
  name: critical-batch
  namespace: sample
spec:
  podSelector:
    matchLabels:
      role: batch-processor
  disruptionPolicy:
    allowedDisruptionWindows:
    - name: weekend-maintenance
      daysOfWeek: [Saturday, Sunday]
      startTime: "00:00"
      endTime: "04:00"
      timeZone: America/Toronto
    minInitialRunDurationDays: 2
    maxNonDisruptionDurationDays: 1
    allowedDisruptionsOutsideOfWindow: [VPA, ClusterAutoscaler]

Map that back to the Part 1 teaser directly:

• Weekday freeze: the allowedDisruptionWindows above name Saturday/Sunday 00:00–04:00 as the only window this workload can be voluntarily disrupted in — every weekday is implicitly frozen.
• Minimum run duration: minInitialRunDurationDays: 2 — a freshly-scheduled pod is untouchable for its first two days, regardless of what the ComputeClass wants to do underneath it.
• Per-workload windows: the guardrail is cluster-wide policy (a platform team’s outer bound — at most 2 windows, at most 30 days without a disruption); the WorkloadClass itself is scoped by podSelector, so critical-batch gets its own rules independent of every other workload in the namespace. A namespace can also carry a workloads.gke.io/default-class label so teams don’t have to annotate every Deployment by hand — the same default-by-label pattern ComputeClasses use. allowedDisruptionsOutsideOfWindow is the pressure valve: name specific sources (VPA, the cluster autoscaler) that can still act outside the window when leaving them fully blocked would be worse than the disruption itself.

That’s the real shape of the split: ComputeClass answers where, WorkloadClass answers when, and neither has to know about the other’s rules to do its job.

Where this series goes

#

Diversity, consolidation, and now disruption scoped to the workload instead of the pool — that’s the flexibility Karpenter’s solver used to hide from you, made explicit. One more post to go:

• Part 3 — Accelerators, reservations, and isolation: GPU/TPU fallbacks (and the GPU toleration gotcha), reservation-first provisioning, Kueue for batch, tenant isolation — and where WorkloadClass and GKE’s open-sourced autoscaling stack leave the Karpenter-vs-ComputeClass portability question by the time this series wraps.