Your company wants to move its coding agents from developer laptops to a shared GKE cluster, for persistence, scale, and governance. Each of those agent sessions stays open all day but works only part of it, and how you run them decides how much of your node bill pays for waiting.
By the end of this article, you will know why an agent session needs a different runtime than a web request, how Agent Substrate and AX run sessions on GKE, and how to identify cost optimization opportunities in your GKE node pools. This article assumes you know Kubernetes Pods and node pools. If not, check out the GKE overview.
Agent sessions wait between prompts
A web request lasts milliseconds. The server answers it, keeps nothing, and serves the next one, so Kubernetes scales a web app by its traffic: more requests, more Pods.
An agent session works another way. A coding agent's workspace can stay open for the developer's whole workday: it keeps its memory and its files between prompts, and it works only after a prompt arrives.
Figure 1. A request ends in milliseconds and keeps nothing. An agent session stays open all day and works in short turns (drawn to scale for 2 prompts an hour).
How much of that day does the agent really work? Much less than it waits. In a study of GitHub Copilot agent traces, Banruo Liu and colleagues found that in a typical session with several prompts, the agent waits for the user about 80% of the time. A turn, one prompt and the agent's whole response to it, takes 396.3 seconds of agent work on average.
Put those numbers on one developer's day. With 2 prompts an hour over an 8 hour workday, which is our assumption, the agent works 22% of the day and waits for the other 78%. If each session holds a machine, you pay for that 78%.
One Pod per session or scale to zero
Teams usually pick one of two answers, and both work well for web apps.
The first is to give each session its own Pod. The session keeps its state, but its Pod holds 1 vCPU and 1 GiB all day. For example, a c3-standard-4 node fits 3 such Pods after GKE's reservations, so in our cost model 100 coding agents need 34 nodes and about USD 1,288 a month, for agents that work 22% of the time.
The second is to scale the Pod to zero between prompts. The node is free, and files on a persistent volume survive, but the process and its memory do not, so the next prompt starts a new agent process.
Based on that, you want a third option: keep the session, and free the machine while the session waits.
Park idle agent sessions in object storage
Agent Substrate and AX are built for that: the developer's session stays open, and the sandbox that serves it gives its machine back while the session waits. Together they cover the parts of an agentic platform that are the same for every team: the control plane, the runtime, and the sandbox. Agent Substrate answers where and how an agent runs. AX answers what the agent is.
Following the CNCF donation of gVisor, Agent Substrate, and OpenRL in November 2026, you have the flexibility to run this open source, agentic platform across any environment. Agent Substrate 1.0 is generally available on GKE.
Figure 2. The building blocks of an agent platform. AX covers the control plane, and Agent Substrate covers the runtime and the sandbox.
One distinction helps from here on. The session is the multi turn interaction between the developer and the agent, with its conversation history. The compute that serves it is an isolated sandbox: an actor in Agent Substrate, or a Task in AX. Parking acts on the sandbox, and the session stays open.
Agent Substrate answers where and how an agent runs. It separates the stateful agent from the compute it runs on. An actor is the sandbox for one agent, and it holds the agent's running processes, files, and identity. A worker is a reusable sandbox drawn from a warm pool of Kubernetes Pods, and one worker hosts several actors.
An agent runs code that a model wrote, so anything in its environment can leak through a prompt injection or a bad command, API keys included. Gateways control the actor's inbound and outbound traffic and add credentials on the way out, based on the actor's identity, so the agent never holds the keys.
SuspendActor snapshots an idle actor to object storage and frees its worker, and the next request restores the actor on any warm worker. Using this approach, we can achieve higher density by multiplexing more agents to run on the same amount of compute.
AX answers what the agent is and what it knows. You declare a Task, which says what runs (image, command, and limits), and a Workspace, which says what the agent knows (repositories and files). AX runs each Task as an actor. ax suspend parks the task and keeps its /workspace. Today AX asks for a volume snapshot, so the files come back on resume and the agent's processes start fresh (runner guide).
AX passes suspend, resume, and delete to Agent Substrate. The AX roadmap adds budgets, policies, and gateways, which limit what an agent may do, and a record of what each task did.
Figure 3. Agent Substrate answers where and how an agent runs, and AX answers what the agent is.
Before you size the pool
Two details decide the bill before you size anything.
First, you decide when a sandbox parks. Agent Substrate does not suspend an idle actor by itself (#424), so your agent harness, or the service in front of it, calls the suspend API after an idle timeout. Until then, an idle actor keeps its worker. When idle actors hold every worker, a request for another actor waits up to 5 seconds by default, then fails with HTTP 503 (Agent Substrate docs).
Second, you decide how much CPU each sandbox is promised. An actor is sized by its CPU and memory limits, and each worker books those limits against the capacity you declare for its pool (API guide). Our cost model sets that capacity to what the node really has, so every actor keeps its full limit. You can declare more capacity than the node has, and then you overcommit CPU at the pool instead of in each Pod.
What parking saves on GKE
Our cost model prices parking for the same 100 coding agents, at Google Cloud list prices in us-central1.
With a 2 minute idle timeout, each session holds a worker 29% of its open time. A warm pool sized for bursts (the 99th percentile of sessions holding a worker at once, if sessions act independently, plus 15% headroom) needs 15 nodes. Add one node that runs around the clock for the Agent Substrate control plane and its PostgreSQL database, and a job that scales the pool to zero after hours:
Parking saves 40% of the node bill. The pool is full about once in 3,500 turns, so a prompt rarely meets that 503. In an hour at 2.5 prompts instead of 2, the same 15 nodes are full for 3% of turns, so size the pool for your busiest hour.
Keep in mind that parking pays most when your agents wait the most. At 2 prompts an hour, the saving is 40%. At 4 prompts an hour, the agent works 44% of the hour, the pool grows to 27 nodes, and the saving is 4%. Around 4.5 prompts an hour, one Pod per session becomes the cheaper option, so measure your prompt rate first.
Also, the pool size is yours to set. The autoscaling option in the GKE install guide follows worker Pods, not suspended actors, so scale the pool to zero after hours with a scheduled job: a pool that runs all 730 hours of the month costs about twice the one Pod per session bill.
Finally, the fleet size matters. The Agent Substrate control plane costs the same for 10 sessions or 1,000, so parking pays from about 37 coding sessions, or 13 for an assumed chat assistant that works about a minute an hour.
Figure 4. Left, the monthly node bill for 100 coding agents against prompts an hour. Right, the cost per session as the fleet grows. Both come from the cost model.
When one Pod per session costs less
Overcommit changes the comparison. If your Pods request less CPU than their limit, Kubernetes places them by their requests. A Pod that requests 0.25 vCPU with a 1 vCPU limit packs 12 to a node, so the same 100 coding agents fit on 9 nodes, fewer than our parked pool. Agent Substrate can overcommit too, through the capacity you declare on the worker pool, and our cost model does not price that case.
Figure 5. The monthly node bill for 100 coding agents by Pod CPU request, from the cost model. Parked sessions keep their full CPU limit in both groups.
Overcommit has the same cost on both sides: agents share CPU and slow down when their neighbors are busy, during builds and tests. Compare like with like before you decide to park: Pods and actors with the same CPU guarantee.
Get started today
Back to the shared cluster. Agents are sessions, not requests: size the warm pool for the sessions that hold a worker, not the sessions that are open.
We would park long lived sessions that wait most of the hour, and keep one Pod per session when developers prompt more than about 4 times an hour. Size the warm pool for working time plus a short idle timeout at your busiest hour, and compare with your current Pods at the same CPU guarantee before you park.
- Take a week of prompts an hour from your gateway logs, and compare your busiest hour with the 4.5 prompts an hour where parking stops paying.
- Install Agent Substrate 1.0 on a GKE cluster with the Agent Substrate on GKE guide.
- Read the Agent Substrate documentation.
- Follow the AX repository. AX is in active development ahead of its first stable release, and the team welcomes your feedback and issues on GitHub.
Thanks for reading! If you run agent sessions on Kubernetes today, how many prompts an hour do your users send? The prompt rate is the input we are least sure of. You can reach me on LinkedIn or X with your numbers, questions, and corrections.
posted in: Previous Next Related Posts
Mobile
Web
Tutorials
How-To Guides
Build Generative UI Apps with A2UI & CopilotKit Agents
OCT. 29, 2026
AI
Cloud
How-To Guides
Best Practices
Catch Go bugs that fuzzing misses with Gemini and a constraint solver
NOV. 12, 2026
AI
Cloud
Tutorials
Case Studies
Unlocking Stable On-Policy RL on TPU with Bitwise-Zero Train–Inference Mismatch
NOV. 16, 2026
AI
Cloud
Case Studies
Influence
How Agent Anomaly Detection scores on benchmarks we didn't write
NOV. 12, 2026