LLM agents: architecture and production operations

An agent for an ML platform might read a failed run, inspect its inputs, propose a preprocessing fix, execute that fix, and compare the result with a baseline. The model decides what to try, but the surrounding platform must supply the execution environment, permissions, resource allocation, and evidence needed to judge the result.

Polyaxon lets teams build this application with the same primitives they use for ML development: services for interactive work, jobs for finite execution, components for reusable definitions, and DAGs for dependent workflows. The agent's reasoning loop lives in application code, while Polyaxon manages the workloads that support it.

Start with a concrete platform task

Consider an agent that helps diagnose failed feature-generation jobs. Its read tools inspect an authorized run and selected artifacts. Its experiment tool executes a proposed calculation in a disposable sandbox. A qualification job checks a candidate patch against fixed input fixtures. A person reviews the resulting report before the patch reaches the shared pipeline.

This task has several lifetimes. A model request lasts seconds; a sandbox may remain available across several attempts; evaluation jobs finish independently; the patch and reports must remain accessible after compute stops. Giving each responsibility an explicit Polyaxon home makes recovery and ownership easier to reason about.

Give interactive tools an explicit environment

Use a sandbox-enabled service when the agent needs to inspect files and run several commands against the same workspace. The following component provides Python and a scratch directory, with a one-hour lifetime and explicit CPU and memory allocation:

kind: component
version: 1.1
name: agent-workspace

plugins:
  sandbox: true
  auth: false
  mountArtifactsStore: false

termination:
  timeout: 3600

run:
  kind: service
  volumes:
    - name: workspace
      emptyDir: {}
  container:
    image: python:3.11
    workingDir: /workspace
    command: ["sleep", "infinity"]
    resources:
      requests:
        cpu: "1"
        memory: "1Gi"
      limits:
        cpu: "2"
        memory: "2Gi"
    volumeMounts:
      - name: workspace
        mountPath: /workspace

This is an execution template, not a complete isolation policy. It disables automatic Polyaxon authentication context and artifact-store mounting because the example's generated calculations do not need them. Platform administrators still configure workload identity, runtime isolation, and network access. Use an approved image digest and organizational presets for a deployed application.

Follow the sandbox quick start to submit the component and wait for the run to reach running. The host application then binds SandboxClient to its owner, project, and UUID and checks ping() before exposing tools. The model should not choose those coordinates.

The workspace persists between commands in the same pod, but each Python command starts a fresh process. Variables do not survive from one execution to another. Files in this emptyDir disappear when the pod is replaced. Download useful results with the filesystem client and register them from a trusted tracked operation, following output persistence.

Move qualification into a reusable job

Once the agent proposes a patch, evaluate it with a job component. Declare inputs for the source revision, patch digest, case manifest, and evaluator revision. Put dependencies in the image so a later run does not depend on packages installed during an interactive session.

The job's code records case-level outcomes under tracking.get_outputs_path(), logs aggregate metrics, and returns a failing process status if a required check fails. Keep artifact paths and numerical acceptance results distinct: a report can exist even when its candidate is rejected.

Register the stable qualification definition as a reusable component in the Component Hub, then use it for manually authored and agent-authored changes. The shared procedure gives both kinds of changes the same evaluation contract.

Orchestrate known dependencies with a DAG

A Polyaxon DAG can prepare fixtures, evaluate candidate configurations, and assemble a comparison report. Use operation dependencies to express ordering, and explicit input or artifact references to carry data between stages. A dependency alone does not transfer files.

The model's conditional tool loop stays inside the agent application. Create a separate operation when the work needs an independent image, queue, resource allocation, retry policy, or retained result. This avoids scheduling a new workload for every small reasoning step while preserving reproducibility for substantial experiments.

Use scheduling presets to provide team defaults for CPU diagnostics, GPU experiments, or protected evaluation environments. Check the effective connections and environment after applying a preset; its configuration becomes part of the candidate's operating context.

Evaluate and recover the complete task

Track whether the agent solved the original feature-generation failure, how many attempts it required, which tools it invoked, and whether its patch passed qualification. Include source, prompt, model, tool, and evaluator revisions in each candidate's tracked inputs. Use run comparison to inspect quality alongside compute time and application-recorded model cost.

Store task state outside ephemeral sandbox memory. On interruption, reconcile the latest command receipt and any external side effects before repeating an action. A queued evaluation job, an active sandbox command, and a completed patch upload require different recovery decisions.

With these boundaries, an agent release becomes a versioned application backed by Polyaxon workloads and evidence. Teams can change its reasoning policy while retaining a consistent way to execute, compare, debug, and qualify the work it produces.