The Polyaxon agent sandbox roadmap
Our plans for persistent sandbox workspaces, suspend/resume, activity-aware idle suspension, and ready pools on Kubernetes—and where AX and Substrate differ.
A coding agent edits a repository, runs tests, and waits for a model response or a person to review its changes. Its files still matter during that wait. Keeping its Pod running also keeps Kubernetes capacity allocated, even when the agent has no work to do.
We are extending Polyaxon sandboxes around that pattern: keep the workspace, release idle compute, and make the next session easier to start. The native roadmap covers persistent workspaces, suspend/resume, activity-aware idle suspension, and an optional pool of ready sandboxes.
These are planned extensions to the existing sandbox service. They are not available lifecycle controls today, and this post does not set a delivery date. The first supported profile will be an opt-in CPU service with one replica, a pinned image, and a dedicated workspace on a supported persistent-volume connection.
What a Polyaxon sandbox does today
A Polyaxon sandbox is a regular kind: service run with plugins.sandbox enabled. The operator creates its Kubernetes workload and Service. The plx-exec daemon runs inside the main container, where it exposes command execution, file access, and terminals.
The sandbox client reaches that daemon through Polyaxon's streams proxy. The proxy authorizes access to the run and supplies its sandbox token before forwarding the request through the run's Kubernetes Service. It strips the caller's platform authorization header and cookies from that forwarded request.
One running service can accept repeated commands. Those commands share the run's image, mounts, connections, and resource configuration. You can already use the usual queues and GPU settings to place a sandbox on an appropriate compute agent—a Kubernetes cluster connected to Polyaxon. The CPU restriction above applies to the initial persistent-workspace roadmap, not to today's sandbox resource options.
Sandbox access does not add another isolation boundary inside the container. The Pod's runtime, permissions, mounts, and network controls determine what the code can access.
Keep the workspace when compute stops
Persistent volumes can already retain files beyond a Pod. The planned lifecycle lets the same sandbox run stop and resume against its dedicated workspace volume. Consider a sandbox that has cloned a repository and written a patch to that workspace:
- Suspend: stop the service workload and scale its Deployment to zero. Retain the run UUID, Kubernetes Service, and workspace volume.
- Wait: keep the files while other workloads can use the released Pod capacity.
- Resume: start a fresh Pod with the same workspace and wait for the sandbox to become ready.
Planned lifecycle: the run and workspace survive; the process does not.
This would let a coding session retain edits between visits without keeping its execution environment running. Files on the supported volume survive. Process memory, running commands, sockets, terminal sessions, and changes made only to the container's root filesystem do not. After resuming, the application starts new commands against the retained files.
The volume remains the working copy. Artifact collection is a separate way to export useful results; it does not capture a live process or automatically preserve every workspace write. Recovery onto another node also depends on the storage system's attachment and topology rules.
Suspension releases the sandbox's Pod allocation. Storage and any nodes kept ready still have a cost. Whether it reduces a cloud bill depends on how the cluster reuses or scales down that capacity.
Suspend only when the work is idle
Polyaxon already supports service culling and termination settings. Culling stops an idle service. The planned suspension action would retain an eligible workspace and run so they can be resumed.
An agent can be busy without sending another HTTP request: a background command may still be compiling code, a terminal may be active, or a file transfer may be in progress. Automatic suspension therefore needs activity from commands, terminals, transfers, and the application itself. Work outside plx-exec needs an application activity signal or a profile that disables automatic suspension.
The planned policy is opt-in. Unknown activity or a failed probe keeps the sandbox running. Before shutdown, Polyaxon must stop admitting new work and recheck activity so an incoming command cannot be silently lost. Health inspection should neither wake a suspended run nor count as user activity.
Explicit suspend/resume comes first. Authorized wake-on-access and automatic idle suspension follow once that lifecycle is reliable. A wake request must still respect the run's permissions and compute quota.
Start matching sandboxes from a ready pool
Persistence helps an existing session. A ready pool addresses a different case: many new sessions using the same approved environment.
For example, a team might launch coding agents from a pinned Python image throughout the day. Instead of waiting for each new sandbox to schedule and initialize, an optional pool would keep a bounded number of unused sandbox runs ready to claim.
Planned pool allocation: claim an unused run once, then replenish with a fresh run.
A claim keeps that run's existing UUID, Service, token, and dedicated workspace. The pool replenishes with a new run. A used sandbox never returns to available inventory. Restarting a container cannot undo every file write, credential exposure, or external action, so this design does not depend on recycling a used environment for another caller.
Pool eligibility must match the approved template and its project, queue, resources, environment, storage, and policy. Caller-specific commands and credentials do not run before allocation. An incompatible request uses ordinary authorized creation or receives an explicit unavailable result; denied access or quota remains an error.
Ready sandboxes consume standby compute and storage. The pool needs limits on inventory, provisioning, and age, with those costs attributed to its owning project and queue. Resuming a suspended run always uses its original workspace; it does not claim a different workspace from the pool.
How this compares with AX and other Kubernetes approaches
AX organizes work into Tasks, Workspaces, and Models and runs each task as an actor on Agent Substrate. Substrate places actors on ready worker Pods, allowing multiple actors to share a worker within its capacity. Kubernetes manages the workers; Substrate handles actor placement and lifecycle. Substrate architecture.
Conceptual topology for single-replica services: a ready sandbox Pod and a shared worker Pod are different units of capacity.
An AX task has no Kubernetes Service of its own. Requests pass through the shared atenet-router, which identifies the target actor and resumes it if needed. Images must implement AX's runner contract. Its documented suspend/resume behavior restores workspace files but starts a new process tree; that differs from a claim that every task continues at the exact instruction where it paused. AX networking, runner contract.
Three different things called a warm pool
“Pre-warmed” can describe three different designs. Each keeps a different part of the execution environment ready:
| Design | What is ready before the request? | What the request receives | Main cost or constraint |
|---|---|---|---|
| Ready sandbox Pod | A complete, unused sandbox environment | A claim on one ready sandbox | Standby Pods consume capacity; caller-specific images, volumes, or credentials may prevent a template match |
| Ready shared worker | A Pod capable of hosting isolated actors | An actor placed or restored on that worker | Workers have finite shared CPU and memory; snapshots, templates, and guest-runtime contracts need management |
| Warm image or node | Cached image layers or available cluster capacity | A newly scheduled workload with fewer cold stages | Pod creation, mounts, and application initialization still occur |
How the tools compare
Polyaxon's planned native pool follows the ready-sandbox design: it prepares ordinary service runs for one-shot claims and replenishes with fresh runs. It does not host several actors inside a worker Pod.
Substrate's WorkerPool follows the shared-worker design. Its templates can prepare a golden snapshot for later actors. That preparation is distinct from keeping workers ready: an unprepared template can still cold-boot. Shared workers are most interesting when many sessions are idle and a smaller fraction need compute at once. They do not remove the capacity needed for continuously active work.
Kubernetes Agent Sandbox exposes SandboxWarmPool and claims for prepared sandboxes. Claim-time environment or volume changes can force a cold start. OpenSandbox's Kubernetes operator also pools ready Pods; its default release strategy deletes the used Pod and replaces it. These are closer to the ready-sandbox approach than to shared actor workers.
OpenShell's Kubernetes deployment uses Kubernetes Agent Sandbox for the workload, alongside a separate supervisor Pod, an authenticated Service boundary, and a PVC for /sandbox. Its documented topology adds access and network-policy controls rather than placing many actors inside a shared worker.
Where a future runtime integration could fit
Our native plan extends the run, operator, proxy, and SDK that Polyaxon users already use. It does not require AX, Substrate, a separate actor scheduler, or a new guest runtime. A cluster could use a stronger isolation profile through an already configured Kubernetes RuntimeClass, provided the required cluster capabilities are available.
Full memory/process snapshots and an optional Substrate integration remain separate research directions. Any future integration would need to preserve Polyaxon run permissions, approvals, quotas, logs, artifacts, and status while adapting execution and state recovery to that runtime. An actor-compatible image and routing contract would be necessary; an arbitrary service image could not be assumed to work unchanged.
Substrate is pre-1.0, and its current API guide says device passthrough to actors is temporarily unsupported. Scheduling a worker on a GPU node therefore does not make that GPU available to its actors. Runtime isolation, snapshot access, and gateway authorization would also need review against the upstream threat model and the deployed implementation.
The decision needs measurements from our workloads: time to the first usable command, pool and template hit rates, standby compute, retained storage, and recovery behavior. Bursts from one image, sessions with different images, and long idle gaps can favor different approaches. We are not claiming a latency or cost improvement before those comparisons.
What we plan to deliver first
The native work is organized around three steps:
- Persistent workspace and explicit suspend/resume: establish the supported CPU profile, storage contract, and shutdown behavior.
- Authorized wake and activity-aware suspension: make returning to an idle workspace convenient without interrupting active work.
- Optional ready pools: accelerate new sessions from matching approved templates, with bounded standby costs. Pooling is a separate optimization and does not block the workspace lifecycle.
For current sandbox workflows, start with creating and controlling a sandbox from Python. Follow the Polyaxon roadmap for availability as this work progresses. If your team needs these changes, share the workflow: how long sessions wait, which files must survive, and whether new sessions usually use the same image. Those details help us choose useful defaults and the next supported profiles.