Roll out an AI agent on Kubernetes with active sessions
Pin sessions to an agent revision, roll out a candidate to new sessions, and retire the previous revision after its work drains.
A support agent asks a reviewer to approve a proposed draft. While it waits, you roll out a new image with a different tool schema. The reviewer clicks Approve an hour later. The new worker can read the conversation, but it may interpret the proposed action differently.
For a change like this, run both revisions during the rollout. Assign new sessions to the candidate while existing sessions continue on the revision that created their pending work. The session store, router, and worker together enforce that choice.
Store the revision with the session
Assign the revision when you create the session and record it in the same durable write as the session ID. A retry of session creation should return the existing assignment. If you have a router and background workers, carry the same revision into queued steps and retries, including a step resumed by an approval callback. Read the revision from trusted session state after checking the caller's access.
Keep a small release manifest behind each revision. Record the image digest, prompt revision, model identifier, tool schema version, and state format version. Pin the model revision where the provider supports it, and record the resolved model when it does not. When the worker calls an external tool, keep a compatible tool contract available for the revisions that still need it. Pinning the image alone cannot preserve a tool whose endpoint changes in place.
Give each revision its own route
Use two Deployments with disjoint selectors, and give each a Service. For r42, put both app: support-agent and agent-revision: r42 in the Deployment's selector and Pod template labels. The r42 Service selects those same labels.
apiVersion: v1
kind: Service
metadata:
name: support-agent-r42
spec:
selector:
app: support-agent
agent-revision: r42
ports:
- port: 8000
targetPort: 8000Keep an equivalent support-agent-r41 Service pointing only at the previous Deployment. An application router in the same namespace can map the stored revision to the corresponding Service name. If agents receive work from queues, enforce the same boundary with revision-specific queues or a dispatcher that sends each task to the right worker. Return a retryable unavailable response if the assigned revision is temporarily down. Silent fallback would let another revision interpret session state it may not understand. Kubernetes Deployments · Kubernetes Services
Kubernetes ClientIP session affinity ties traffic to a backend based on the client address. It cannot distinguish two agent sessions from the same client, recover a revision after the original Pod disappears, or route a background job. Store the application session assignment even if the transport also uses affinity. Service session affinity
Change the default for new sessions
- Start
r42and verify its worker can read its session state, call the permitted tools, and resume an interrupted step. Run the same evaluation cases againstr41andr42, including sessions paused for approval. Track completed sessions, tool validation failures, and resume failures by revision. - Assign a small cohort of new sessions to
r42. Keep that decision for the life of each session, even if you change the rollout percentage. Let existingr41sessions finish onr41. - If
r42fails, switch new sessions back tor41. For sessions already assigned tor42, decide whether to complete them there, pause them, or migrate them with a reviewed state conversion. A migration involving a pending tool action needs to reconcile that action and recheck authorization before continuing. - Retire
r41after sessions, queued retries, and pending approvals assigned to it have finished or reached an explicit expiry. A worker pool with zero active requests can still have sessions waiting to resume. Preserve the release manifest and evaluation record for investigation.
Sharing the session database requires a compatible schema during the overlap. Add fields both revisions can tolerate and defer destructive changes until you have drained the old one. Session pinning preserves the expected program and state format. Current access policy still applies when an action resumes.
Make the rollout check concrete. Create a session on r41, pause it for approval, switch the default to r42, and replace the Pod that handled the first turn. After approval, another r41 Pod should resume that session. Create a fresh session and verify it gets r42. Then switch the default back and verify the already assigned r42 session still routes to r42.
Keep evaluation evidence with the release
Run the same evaluation job in Polyaxon for each candidate. Record the agent revision, image digest, evaluator revision, dataset version, and result artifacts against the run. Include a paused session and a worker restart in the cases so a gain on fresh requests cannot hide a resume regression. Polyaxon tracking · Logging artifacts
If you host the agent on Polyaxon, deploy the revisions as separate agent service runs and have your authenticated router map each revision to its configured endpoint instead of the example Kubernetes DNS names. Record the service run UUID with the release manifest. The router and session store own assignment, rollback, and retirement, regardless of where the services run.