Google retools Kubernetes for agentic workloads

Agentic workloads are pushing Kubernetes into territory its creators never planned for. Agents generate untrusted code on the fly, sit idle for long stretches and churn through pods at a pace that strains the orchestrator’s control plane.

Cloud providers are now rethinking how general-purpose a cluster should be. Google LLC’s answer is to stop treating every cluster as an infinitely flexible engine and tune each one into a runtime optimized for its workload, according to Jago Macleod (pictured, right), director of engineering for Kubernetes and Google Kubernetes Engine at Google Cloud.

“There isn’t enough memory. There aren’t enough machines. So we have to get more efficient. And so one of the ways that we do that is to focus the Kubernetes cluster on the workload it’s running, and optimize it for that specific workload,” Macleod said.

Macleod and Jaana Dogan (pictured, left), principal engineer at Google Cloud, spoke with theCUBE’s Rebecca Knight at KubeCon + CloudNativeCon NA, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed how Google is customizing Kubernetes for agentic workloads without fragmenting the open-source ecosystem. (* Disclosure below.)

Agentic workloads strain the Kubernetes control plane

Kubernetes was built for microservices, then stretched to handle training jobs in which a single pod claims an entire node, Macleod explained. Agents are a different beast again, and some teams have resorted to building custom control planes just to cope.

“I think the agentic workloads are so different in the thrash and the churn that they represent is really not what we built Kubernetes for at all,” Macleod said. “And then agentic applications are almost the polar opposite, which is, you know, in the naive implementation, you just have as many pods as you can possibly fit, and so then the API machinery is running into limits.”

Platform teams have spent a decade making Kubernetes their compute fabric, and customers told Google they did not want agents to fragment it, Dogan explained. Google’s response is Agent Substrate, which runs AI agents in sandboxes directly on Kubernetes. The company donated the project to the Cloud Native Computing Foundation and recently released version 1.0. Because agents write their own code, that code is untrusted by default and needs sandboxing and network guardrails, according to Dogan. Customers also flagged a costly inefficiency.

“For example, our customers came to us, ‘Hey, majority of the time our agents are idle, but we have to allocate a sandbox all through the lifetime of a session,’” Dogan said. That means wasting a lot of CPU cycles, she explained.

Coding agents are also changing how engineers work, producing more prototypes and smaller, more focused teams, Dogan noted. As agentic workloads multiply, human judgment grows more valuable rather than less, according to Macleod.

“It’s not really, you know, fire off a prompt and forget it and walk away,” Macleod said. “It’s just a different level in the system where you’re applying your judgment and expertise. And that has never been more important.”

(* Disclosure: Google Cloud sponsored this segment of theCUBE. Neither Google Cloud nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE