Choose an AI agent architecture for production

An agent architecture determines which decisions are fixed in code, which are delegated to a model, and how the application checks the result. That choice affects latency, operating cost, permissions, and the work required to investigate a failure.

For a Polyaxon deployment, there is a second design choice: which parts run as finite jobs, which need a service, and which have stable dependencies suitable for a DAG? Keep that workload structure separate from the framework's internal reasoning and tool loop.

Start with a task whose successful completion can be observed. Then locate the uncertainty that the model needs to resolve.

Identify the variable part of the task

Consider a service that investigates failed data imports and prepares an incident brief. The application must always load the authorized incident record, produce an evidence-backed diagnosis, and send the result to the assigned owner.

Some incidents follow a known path: inspect the schema validation result and identify a missing field. Others require an adaptive investigation across connector status, version changes, and related logs.

Those two populations may need different execution paths. Estimate their frequency and consequence before choosing a general agent loop for all requests.

Match the control pattern to the uncertainty

Anthropic's architectural distinction between workflows and agents separates predefined control paths from model-directed execution. A practical application can combine both.

For the import service, begin with a fixed path for schema errors. Add routing when incident categories have sufficiently different handling requirements. Use a bounded investigation loop for cases whose next useful query cannot be known in advance.

Design the fixed path as the baseline

A baseline might read the validation report, retrieve matching documentation, generate a diagnosis, and validate the output structure and references. It can provide useful service without choosing tools dynamically.

Preserve ambiguous cases instead of forcing them through this path. Record why the workflow could not complete them: missing evidence, unknown error type, conflicting records, or unavailable dependencies.

These cases define what a more flexible architecture must improve. Without a baseline and failure taxonomy, an agent loop can add work while leaving the original problem unchanged.

Make routing recoverable

If a classifier selects the incident path, validate its output against a small route set. Provide an explicit unknown route, and allow downstream evidence to reject an inappropriate route.

An incident initially labeled as a schema error might actually result from a connector returning truncated data. If the schema workflow finds no matching evidence, it should return a typed unresolved result rather than manufacture a diagnosis.

Keep route-specific access narrow. Choosing a route is not an authorization decision; the execution layer still checks whether the task may read the relevant systems or perform an action.

Measure misrouting and recovery separately. An application that recovers from every wrong route may remain correct while consuming unnecessary calls and time.

Bound the adaptive investigation

A tool loop needs a stopping contract. Define the maximum steps, task deadline, available tools, and conditions for completion or escalation. Return structured tool results that distinguish missing data, denied access, temporary failure, and successful reads.

For the import service, completion might require a diagnosis category, supporting evidence identifiers, and unresolved uncertainties. Verify those outputs against the incident record and source material. The model's declaration that the investigation is complete is a proposal for the application to validate.

Detect lack of progress using observable behavior, such as repeated equivalent queries or an unchanged evidence set. A stalled investigation should produce an actionable escalation with the work already collected.

The durable execution guide covers what must survive a worker interruption. The architecture decision here concerns when adaptive exploration is worth its added execution and verification cost.

Run a bounded investigation as a job

A finite investigation with a known input and a result artifact can use a Polyaxon job. This component template assumes an image containing your import_investigator module, its agent framework, and a sanitized incident fixture. The module implements --max-steps and --deadline-seconds; those are application options, not built-in agent settings in Polyaxon.

version: 1.1
kind: component
name: investigate-import-failure
termination:
  timeout: 900
  maxRetries: 0
run:
  kind: job
  container:
    image: registry.example.com/team/import-investigator:release-17
    command: [python3, -m, import_investigator]
    args:
      - "--incident=/app/fixtures/import-042.json"
      - "--max-steps=8"
      - "--deadline-seconds=600"
    resources:
      requests:
        cpu: "1"
        memory: 2Gi
      limits:
        cpu: "2"
        memory: 4Gi

The application deadline leaves time to save a final result before the illustrative workload timeout. Your program must enforce per-call timeouts as well; a blocked tool call should not prevent it from observing its own deadline. Termination settings control the workload's lifetime and retry behavior, while the framework controls the loop and any checkpoint state.

Have the runner use tracking to log its configuration and numeric outcomes, and save the diagnosis, evidence references, and unresolved questions as artifacts. Use connections for required store or provider access after replacing the illustrative image.

Use a service for an interactive endpoint

An interactive application or reviewer dashboard needs a process that continues accepting requests. Polyaxon's service runtime supports APIs, internal tools, and dashboards. The runtime excerpt below assumes your image contains an incident_review_api module that listens on the supplied host and port:

run:
  kind: service
  ports: [8000]
  container:
    image: registry.example.com/team/incident-review:release-17
    command: [python3, -m, incident_review_api]
    args: ["--host=0.0.0.0", "--port=8000"]

Configure the service's access, persistence, resources, and health behavior for your deployment. Exposing a port is not a complete public production API configuration. The application still needs request identity, authorization, admission limits, and a durable place for task state when continuation is required.

For batch qualification, keep using jobs even if the released application is a service. A DAG can prepare incident fixtures, invoke a candidate endpoint, score its results, and produce a report. There is no need to make each internal model call a separate infrastructure operation.

Delegate only when the work has a useful boundary

Suppose an incident spans three independently operated connectors. Separate investigators could inspect each connector concurrently and return a bounded evidence bundle. A coordinator can then compare their findings.

Define who owns shared state, how conflicting findings are resolved, and what happens when only some investigators finish. Preserve each result's source and scope. Agreement between agents using the same misleading evidence is not independent confirmation.

Avoid handing every agent broad credentials for convenience. Assign only the resources needed for its subtask, and keep consequential writes under a single authorized owner.

Compare this design with a single investigator on the same incidents. Include communication, duplicate retrieval, result synthesis, partial failures, and additional model calls in the measurement. Parallelism can shorten elapsed time while increasing total work.

Compare architectures through Polyaxon

Package each candidate behind the same task input and result contract. Expose an architecture input on the evaluation component and compare fixed, routed, and bounded_loop implementations using the grid-search pattern. Those values select your application code paths; they do not change Polyaxon's scheduler into an agent framework.

Keep dynamic tool loops and agent coordination within the application or framework that implements them. The outer workflow can coordinate independent evaluation workloads without redefining the framework's live state semantics.

Record task completion, supported diagnoses, prohibited actions, escalation, elapsed time, and total resource use with tracking. Retain trajectories and evidence bundles as protected artifacts for case-level inspection.

In the runs comparison dashboard, inspect results by incident category as well as overall score. Use the monitoring view for workload-level delays and resource consumption, and your application records for tool and provider timing. This distinguishes an expensive control pattern from an otherwise useful agent waiting on its environment.

Choose the architecture that improves the required task population within the application's constraints. Keep simpler paths for cases they already handle well, and expand model-directed behavior where its benefit can be demonstrated.