Polyaxon v3 is coming →

Roll out an AI agent on Kubernetes with active sessions

Pin sessions to an agent revision, roll out a candidate to new sessions, and retire the previous revision after its work drains.

November 21, 2026by Polyaxon

A support agent asks a reviewer to approve a proposed draft. While it waits, you roll out a new image with a different tool schema. The reviewer clicks Approve an hour later. The new worker can read the conversation, but it may interpret the proposed action differently.

For a change like this, run both revisions during the rollout. Assign new sessions to the candidate while existing sessions continue on the revision that created their pending work. The session store, router, and worker together enforce that choice.

A returning session uses its stored r41 assignment and reaches the r41 service. A new session receives r42 at creation and reaches the r42 service. Both routes remain available during the rollout.

Store the revision with the session

Assign the revision when you create the session and record it in the same durable write as the session ID. A retry of session creation should return the existing assignment. If you have a router and background workers, carry the same revision into queued steps and retries, including a step resumed by an approval callback. Read the revision from trusted session state after checking the caller's access.

Keep a small release manifest behind each revision. Record the image digest, prompt revision, model identifier, tool schema version, and state format version. Pin the model revision where the provider supports it, and record the resolved model when it does not. When the worker calls an external tool, keep a compatible tool contract available for the revisions that still need it. Pinning the image alone cannot preserve a tool whose endpoint changes in place.

Give each revision its own route

Use two Deployments with disjoint selectors, and give each a Service. For r42, put both app: support-agent and agent-revision: r42 in the Deployment's selector and Pod template labels. The r42 Service selects those same labels.

apiVersion: v1
kind: Service
metadata:
  name: support-agent-r42
spec:
  selector:
    app: support-agent
    agent-revision: r42
  ports:
    - port: 8000
      targetPort: 8000

Keep an equivalent support-agent-r41 Service pointing only at the previous Deployment. An application router in the same namespace can map the stored revision to the corresponding Service name. If agents receive work from queues, enforce the same boundary with revision-specific queues or a dispatcher that sends each task to the right worker. Return a retryable unavailable response if the assigned revision is temporarily down. Silent fallback would let another revision interpret session state it may not understand. Kubernetes Deployments · Kubernetes Services

Kubernetes ClientIP session affinity ties traffic to a backend based on the client address. It cannot distinguish two agent sessions from the same client, recover a revision after the original Pod disappears, or route a background job. Store the application session assignment even if the transport also uses affinity. Service session affinity

Change the default for new sessions

  1. Start r42 and verify its worker can read its session state, call the permitted tools, and resume an interrupted step. Run the same evaluation cases against r41 and r42, including sessions paused for approval. Track completed sessions, tool validation failures, and resume failures by revision.
  2. Assign a small cohort of new sessions to r42. Keep that decision for the life of each session, even if you change the rollout percentage. Let existing r41 sessions finish on r41.
  3. If r42 fails, switch new sessions back to r41. For sessions already assigned to r42, decide whether to complete them there, pause them, or migrate them with a reviewed state conversion. A migration involving a pending tool action needs to reconcile that action and recheck authorization before continuing.
  4. Retire r41 after sessions, queued retries, and pending approvals assigned to it have finished or reached an explicit expiry. A worker pool with zero active requests can still have sessions waiting to resume. Preserve the release manifest and evaluation record for investigation.

Sharing the session database requires a compatible schema during the overlap. Add fields both revisions can tolerate and defer destructive changes until you have drained the old one. Session pinning preserves the expected program and state format. Current access policy still applies when an action resumes.

Make the rollout check concrete. Create a session on r41, pause it for approval, switch the default to r42, and replace the Pod that handled the first turn. After approval, another r41 Pod should resume that session. Create a fresh session and verify it gets r42. Then switch the default back and verify the already assigned r42 session still routes to r42.

Keep evaluation evidence with the release

Run the same evaluation job in Polyaxon for each candidate. Record the agent revision, image digest, evaluator revision, dataset version, and result artifacts against the run. Include a paused session and a worker restart in the cases so a gain on fresh requests cannot hide a resume regression. Polyaxon tracking · Logging artifacts

If you host the agent on Polyaxon, deploy the revisions as separate agent service runs and have your authenticated router map each revision to its configured endpoint instead of the example Kubernetes DNS names. Record the service run UUID with the release manifest. The router and session store own assignment, rollback, and retirement, regardless of where the services run.