Manage long-lived AI agent workspaces
Manage long-lived Polyaxon agent workspaces with explicit ownership, reproducible environments, checkpoints, bounded lifetimes, and safe replacement.
A long-lived workspace can make a coding assistant more useful: the repository is already present, intermediate files remain available, and the engineer can return to inspect the work.
The same persistence can accumulate hidden state. Package installations, temporary credentials, and manual changes can turn a convenient session into an environment that nobody can reproduce.
Operate long-lived Polyaxon services as managed sessions, not permanent machines.
Define ownership and an end condition
Associate each workspace with an authorized user or application session, a project, and a purpose. Record who may reconnect, what data it contains, and when it should be retired.
Use sandbox access through the platform's authenticated interfaces. Keep any application-level tenant authorization in the controller that maps the session to its run.
Even a multi-day task should have an absolute lifetime and an owner responsible for extension or replacement. “Until somebody remembers to stop it” is not an operating policy.
Keep the baseline outside the workspace
Create the service from a reviewed component with a pinned image revision, resource profile, connections, and workspace mounts. Save dependency changes in the source repository or image build definition.
A useful workspace may contain uncommitted edits, but those edits should not be the only copy of important work. Periodically export a patch or checkpoint through the persistence workflow.
Distinguish a persistent volume from a persistent process. A volume can retain files after replacement without preserving shell sessions, open connections, or Python memory.
Checkpoint at meaningful milestones
A checkpoint for a coding agent can include:
- Base source commit and proposed patch.
- Dependency or image revision.
- Input dataset and configuration references.
- Approved reports and execution receipts.
- Remaining task description.
Exclude credentials, unnecessary caches, and unrelated user files. A trusted collector should validate what is copied out of an environment running generated code.
Store durable evidence using Polyaxon artifacts. Logging a file reference does not copy scratch data automatically; use the synced outputs path or retrieve the file for trusted persistence.
Replace the environment before drift becomes normal
Plan a periodic reconstruction from the recorded baseline and latest checkpoint. This reveals missing dependencies and allows the workload to receive reviewed image updates.
Before stopping an active workspace, reject new requests, finish or reconcile running commands, and preserve useful outputs. Then start the replacement and verify the expected files and application state.
Make replacement a short, repeatable runbook:
- Mark the session as draining in the controller so reconnects cannot start more commands.
- Record active command handles and determine which completed, which stopped, and which remain uncertain. Do not repeat an uncertain external write to make the workspace look healthy.
- Save a checkpoint manifest containing the source/image/input revisions, patch or report references, and remaining task. Confirm the referenced files are durable before advancing its state revision.
- Request termination and retain the cleanup receipt. Keep the old run visible until termination is confirmed.
- Create a fresh service, restore the approved files, and run a task-specific readiness check. Bind the existing logical session to the new run only after that check succeeds.
Keep the old and new run UUIDs in the replacement record. The state-lifetime matrix identifies what to restore; the benchmark fixture offers a small counter example for verifying the difference between session reuse and a new workspace. A valid counter is only a fixture check, not evidence that a real repository or agent state was restored correctly.
Use termination settings and supported service culling to bound forgotten sessions. Culling terminates idle services; it is not a memory snapshot or a guarantee of instant resume.
Measure active value and accumulated risk
Track useful command time, total session duration, checkpoint age, and reconstruction failures. A workspace that stays idle for days may be cheaper and safer to recreate.
Review connections when the task changes. A session that began as repository inspection should not silently retain a broad credential after the relevant operation is finished.
Long-lived execution can be a practical product feature without becoming indefinite infrastructure. Polyaxon supplies the service and run context; reproducible baselines and deliberate checkpoints keep the workspace manageable.