Build a reliable foundation for autonomous agents
Build an operational foundation for autonomous agents using Polyaxon workload profiles, durable state, bounded authority, evaluation, and recovery ownership.
An autonomous agent can continue making decisions after the initial user request. That makes the operating foundation important: the system needs a budget, a durable record of progress, and a way to stop or recover work without relying on the model's judgment.
Build that foundation around a small set of reviewed Polyaxon workload profiles and application contracts.
Start with an operational agreement
For each agent capability, document its permitted inputs, tools, data destinations, maximum runtime, and completion criterion. Assign an owner for incidents and configuration changes.
An investigation agent may read a failure bundle and propose a patch. A deployment agent may have a different approval path and stronger audit requirements. Do not give the first agent the second agent's credentials merely because both use the same framework.
Use project permissions and connections to keep execution access aligned with the intended role.
Standardize the execution profiles
Define a finite job profile for preparation and evaluation, a service profile for the controller API, and a sandbox-enabled service profile for interactive commands.
Package reviewed images, resources, timeouts, and initialization behavior as components. Use presets for shared defaults while enforcing mandatory restrictions through the appropriate platform and cluster controls.
Where available, queues and compute agents route workload classes to suitable capacity. Keep interactive sessions and large background experiments from competing without a deliberate policy.
Persist intent and outcome outside the executor
The controller should record an accepted request before dispatch and associate it with the resulting run UUID. Preserve enough state to determine whether work is pending, complete, or uncertain after a restart.
Store useful outputs through Polyaxon artifacts. Keep mutable session state in an application-managed store rather than relying on a container's memory or scratch directory.
An interrupted request should be recoverable from recorded inputs and checkpoints. If it cannot be reconstructed, identify the missing state before increasing autonomy.
Make evaluation part of normal operation
Use a trusted evaluator for the task's completion criterion. A zero exit code indicates process completion, not that the agent met the user's objective.
Track task quality, denied actions, retries, and resource consumption with Polyaxon tracking. Preserve representative failures so improvements can be compared against the same cases.
Keep the evaluator and promotion credentials outside the generated program's writable environment. The agent may propose another attempt, but it should not redefine the threshold that approves its own work.
Define intervention and recovery
Provide an operator path to stop accepting new requests, terminate active work, revoke affected credentials, and inspect the evidence. Make ownership clear across the application, platform, and infrastructure teams.
Use termination controls to bound individual workloads. Add cumulative budgets in the controller for total steps and parallel tasks.
Exercise a small set of failure scenarios before expanding rollout: unavailable dependency, lost client connection, replaced executor, and incomplete output collection. Confirm that each produces an understandable state.
Use the bounded evaluation adapter and workspace lifecycle as a small operational rehearsal. Assign owners to the resulting records:
| Observed condition | First action | Evidence required before resuming |
|---|---|---|
| Submission receipt is uncertain | Stop automatic resubmission and inspect the correlation tag | A verified run-to-request mapping or an explicit operator disposition |
| Readiness deadline expires | Attempt cleanup and retain the run UUID | Terminal status or a named owner for unconfirmed termination |
Evaluator reports invalid_input | Repair the input/manifest contract | Complete coverage of the authoritative cases |
| Evaluator rejects a complete candidate | Return the decision to the application owner | A reviewed new candidate or accepted exception |
| Workspace is replaced | Restore from the approved checkpoint | Matching source, input, and image revisions in the new run |
This runbook gives each result an owner without treating every failure as a model problem. The execution-contract example supplies the identifiers to carry into an incident, and the durable recovery sequence covers the case where an external action may already have completed.
A reliable foundation is not a promise that the agent never fails. It is a system in which failures remain bounded, visible, and recoverable while successful work becomes reusable platform evidence.