Kubernetes for AI agents: A platform engineering guide

An AI agent may receive a task through a service, retrieve context, call several tools, execute code in an isolated environment, wait for approval, and resume hours later. Kubernetes can place and replace the processes behind that path, but it does not automatically understand the logical task or whether an external action already succeeded.

That is the platform engineering challenge: use Kubernetes for the responsibilities it handles well, then add an agent execution layer for the application semantics it cannot infer. The result should give developers a simple path to production without hiding the security, state, and recovery decisions that determine whether an agent is safe and reliable.

Treat Kubernetes as the execution substrate

Kubernetes provides a mature control plane for containers, networking, identity, storage attachment, scheduling, and reconciliation. Those primitives are valuable for agent workloads because their demand can be bursty, their execution may need isolation, and their components often have different resource profiles.

The substrate does not own the meaning of an agent task. A replacement Pod does not know which reasoning step was committed, whether a tool call may be repeated, or which prompt and policy revisions produced an earlier decision. Keep those concerns in an application runtime or control plane with durable records.

This division avoids two common mistakes: putting application recovery logic into deployment scripts, and assuming that an agent framework replaces the operational controls of the cluster.

Classify the workload before choosing a controller

“AI agent” is not one execution shape. Classify each component by its lifetime, concurrency, state, and latency requirements.

The Kubernetes workload documentation recommends using a workload resource to manage Pods instead of creating unmanaged Pods directly. The controller expresses the desired lifecycle; the Pod remains a replaceable execution unit. The next article in this series compares Pods, Jobs, Services, and sandbox environments for agents.

Build a golden path around an execution contract

A useful internal platform turns a small set of decisions into a repeatable submission path. Ask the developer for the application image, command, resource needs, expected lifetime, input and output references, required connections, and trust level. Apply organization policy through versioned templates or presets.

The resulting execution contract should identify:

• the logical task and execution attempt;
• the code, image, prompt, tool, and policy revisions;
• the CPU, memory, GPU, storage, and placement requirements;
• the identities and outbound destinations allowed to the workload;
• the deadline, retry policy, and application budget;
• the state and artifacts that must survive replacement; and
• the evidence required to accept the result.

Keep the contract inspectable after submission. A platform is not self-service if operators must reconstruct the effective configuration from admission mutations, implicit defaults, and several unrelated dashboards.

Scope identity to the work being performed

An agent's authority includes every credential, mounted file, reachable service, and delegated user permission available to its process. Prompt-level instructions are not an access-control boundary.

Use a dedicated Kubernetes ServiceAccount and the narrowest practical RBAC permissions for each workload class. Avoid mounting Kubernetes API credentials when the container does not need them. Prefer short-lived credentials and workload identity mechanisms over static keys copied into images or broad shared secrets.

Separate platform identity from tool authorization. A ServiceAccount may authenticate the workload to infrastructure, while the application still needs to decide whether this task may invoke a payment API, modify a repository, or read a customer record. Record both decisions with the task.

Apply NetworkPolicy as part of the execution contract, starting from deny-by-default where the cluster networking implementation supports it. Test DNS, model endpoints, artifact storage, telemetry, and required tools from the actual workload identity. A policy manifest that was never exercised is not evidence of isolation.

Make state survive infrastructure changes

Agent state may include a conversation, a workflow checkpoint, retrieved evidence, temporary files, approval decisions, and receipts from external actions. These records do not all need the same storage or retention policy.

Keep the source of truth for task progress outside the lifetime of a single Pod. Persist large artifacts in approved object or volume storage, and keep coordination state in a backend with the consistency required by the recovery protocol. A replacement worker should load a specific committed revision rather than guess what the previous process completed.

Waiting for a human should usually release expensive execution capacity. Persist the pending action, its exact arguments, the required approver, and its expiration. Resume through a new execution attempt when a decision arrives. The durable execution guide explains how to combine checkpoints, idempotency, and action reconciliation.

Observe infrastructure and agent behavior together

Cluster telemetry answers whether a container was scheduled, restarted, throttled, or unable to reach a dependency. Agent telemetry answers which model, context, tool, state transition, and policy decision produced an outcome. Production diagnosis needs both layers connected by stable identifiers.

Record the logical task ID, attempt, application version, and tenant or project context with logs, metrics, traces, and retained artifacts. Avoid putting prompt content, credentials, or sensitive tool results into unrestricted labels or logs. Apply redaction and retention deliberately.

Infrastructure health is not task success. Track accepted outcomes, policy violations, queue delay, time to first useful response, tool failures, model cost, and recovery attempts alongside CPU, memory, GPU, and network signals. The AI agent tracing guide covers the trajectory layer, while agent SLOs turn those signals into user-centered objectives.

Control resource and dependency pressure

Requests and limits affect placement and isolation, but local compute is only one source of pressure. Agent throughput can be constrained by model-provider limits, database connections, tool concurrency, or GPU memory in a shared inference service.

Apply admission, queueing, priority, and quota by workload class. Then add application-level backpressure for external dependencies. Scaling a Deployment or dispatching more Jobs should not multiply retries against an already saturated provider.

Measure completed, accepted work per resource-hour and per provider dollar. Idle warm workers may be justified by an interaction SLO; idle task Pods waiting on approval usually are not. The right capacity model follows measured demand and recovery time, not the agent label.

Apply the model with Polyaxon

Polyaxon provides tracked jobs, services, DAGs, connections, artifacts, resource configuration, and execution history on Kubernetes. Use a reusable component to package the workload and an operation to apply inputs, scheduling, and policy for a particular run.

Platform teams can encode common node selection, security context, connections, and resource settings in presets. Commercial queues add priority, concurrency, quota, and routing controls across execution environments. Those controls govern workload dispatch; the agent application still owns its task checkpoint, tool authorization, and side-effect semantics.

Use configured artifact connections for retained outputs and link them to the execution record. For interactive development, Polyaxon sandboxes expose process and filesystem access inside a service run; their permissions and isolation remain those of that workload configuration.

Review one task end to end

Before onboarding many teams, follow one representative task through acceptance, queueing, scheduling, execution, tool access, interruption, recovery, and completion. Confirm that an operator can answer:

1. Which task and tenant does this process represent?
2. Which code, prompt, model, tool, and policy versions are active?
3. Why was this workload admitted and placed here?
4. Which credentials, files, and destinations can it access?
5. What state survives if the Pod or node disappears?
6. How is an uncertain external action reconciled?
7. Which evidence proves that the final result was acceptable?

Kubernetes gives agent platforms a strong execution foundation. The production platform emerges from the contracts around it: explicit workload shapes, durable task state, least-privilege authority, bounded resources, and evidence that connects infrastructure behavior to agent outcomes.