Choose execution patterns for AI agents with Polyaxon

An AI agent that classifies a batch of reports needs a different execution pattern from an agent that edits a repository over several tool calls. In Polyaxon, you can give the first task a finite job and the second a sandbox-enabled service, while keeping their configuration, logs, outputs, and evaluation results in the same platform.

This is a practical starting point for teams considering serverless execution. Define when each piece of work starts, what must remain available between requests, and when its resources can be released. Polyaxon provides the workload and orchestration controls; elastic node provisioning, provider billing, and model endpoint scaling depend on the infrastructure you connect to it.

Map the agent to Polyaxon workloads

Consider a coding assistant that investigates failed training runs. Its controller selects a failure report, asks a model to propose a change, executes commands, and evaluates the result. Represent the infrastructure needs separately:

Use jobs for work that finishes and services for processes that need to remain available. A model's reasoning loop can stay in your application; every internal decision does not need its own infrastructure operation.

Package finite tasks as reusable jobs

Build a component for the repeatable part of the task: its image, inputs, command, outputs, and resource requirements. For example, an evaluation worker could use the following component. Replace the example image with your versioned image containing /app/evaluate.py and the Polyaxon tracking library:

version: 1.1
kind: component
name: agent-evaluation
inputs:
  - name: manifest
    type: str
termination:
  timeout: 1800
  maxRetries: 0
run:
  kind: job
  container:
    image: registry.example.com/ml/agent-evaluator:1.0
    command: ["python", "/app/evaluate.py"]
    args: ["--manifest={{ manifest }}"]
    resources:
      requests:
        cpu: "1"
        memory: "2Gi"
      limits:
        cpu: "2"
        memory: "4Gi"

Here, manifest identifies the input your worker knows how to retrieve. Configure any required storage access through approved connections. The timeout bounds the operation; the worker should also set request and tool-call timeouts so a stalled dependency does not consume the entire allowance.

Start without retries for tasks that may change external systems. Add retries only after the worker can recognize completed actions and avoid repeating them. A classification job and a deployment job may need different retry policies even if both call the same model.

Once the component is stable, publish a component version. An operation can supply inputs, select a queue, apply a resource preset, and add scheduling without duplicating the worker definition.

Keep an interactive workspace only as long as needed

For a coding session, follow the sandbox quick start to launch a service with plugins.sandbox: true. The sandbox plugin provides process, file, and terminal access inside that service's main container. Its permissions and isolation come from the service environment and cluster configuration.

The controller waits for the run to reach RUNNING, checks sandbox.ping(), and then uses SandboxClient for commands and file transfer. Files in /workspace can connect successive tool calls while the same service remains running. Separate Python command executions do not share in-memory variables.

Keep model credentials in the controller when following the LLM integration workflow. Give the service only the connections and files its commands require. This also makes it easier to replace the model provider without rebuilding the execution environment.

Make resource release an application behavior

Stop a sandbox service when its task finishes, after saving its useful outputs. An absolute termination.timeout provides a backstop when the controller fails to clean up. From Polyaxon v2.12, services can also use termination.culling with an activity probe to stop after an idle period. See the termination specification for the distinction between runtime timeout, idle culling, and retention after completion.

Culling stops the service. To continue later, the application must recreate its environment and restore the files or task state it deliberately saved. Use the outputs workflow for durable artifacts and Git for retained code changes.

Releasing a run frees its allocated resources for other workloads. Whether that reduces the infrastructure bill depends on node utilization, autoscaler behavior, and your compute agreement. Measure those effects separately from the service's active duration.

Compare execution policies in the runs dashboard

Use the same task manifest to compare a fresh service per task, a service retained for one coding session, and a finite job for a known command sequence. Record setup time, useful execution time, task success, retries, and retained capacity. Separate image pulls, data preparation, and model request latency so an apparent sandbox startup problem can be traced to its actual source.

Log measurements with Polyaxon tracking and inspect candidates in the comparison dashboard. Preserve case-level reports for timeouts and failed tasks as well as successes. The useful decision is the execution policy that meets your completion and latency requirements at an acceptable total cost.

As the agent becomes reliable, move recurring work into scheduled operations or a DAG. Polyaxon then carries the same component definitions from interactive development into repeatable production workflows.