Deploy AI agents as Polyaxon services

Deploying an AI agent means turning a development loop into an application that can accept requests, recover from failures, and explain what happened. The model call is only part of that application.

A Polyaxon service can host your agent API, while separate jobs or sandbox-enabled services perform evaluation and code execution. Keep these responsibilities distinct so a generated program does not inherit the API server's credentials.

Package the application contract

Build an image containing your reviewed agent application, locked dependencies, and startup command. Decide whether it serves short synchronous requests or returns a task identifier for longer work.

Your application should define its own authentication requirements, request limits, durable session state, and cancellation behavior. Polyaxon's authenticated service proxy controls platform access; it does not implement your product's customer-level authorization model.

For long-running requests, store progress outside process memory. A replacement Pod should be able to determine whether the task completed, failed, or requires reconciliation.

Declare a service and its health endpoints

This component assumes you have built registry.example.com/team/agent-api:reviewed with an app.main module and implemented /health/ready and /health/live. Replace the illustrative image reference with your approved digest.

kind: component
version: 1.1
name: agent-api

run:
  kind: service
  ports: [8000]
  container:
    image: registry.example.com/team/agent-api:reviewed
    command:
      - uvicorn
      - app.main:app
      - --host
      - "0.0.0.0"
      - --port
      - "8000"
    resources:
      requests:
        cpu: "1"
        memory: 1Gi
      limits:
        cpu: "2"
        memory: 2Gi
    readinessProbe:
      httpGet:
        path: /health/ready
        port: 8000
      initialDelaySeconds: 5
      periodSeconds: 10
    livenessProbe:
      httpGet:
        path: /health/live
        port: 8000
      initialDelaySeconds: 20
      periodSeconds: 20

The image must include Uvicorn and your application. A readiness check should indicate whether the application can accept work; liveness should detect a stuck process without restarting it merely because a model provider has a temporary outage.

Review service configuration and network access for your deployment's proxy and path behavior.

Supply credentials deliberately

Attach configured connections only to the workloads that require them. The agent API may need a model-provider credential and a session database. A generated-code executor may need neither.

Where available in your Polyaxon edition, use queues to route production services separately from evaluation and development workloads. Queue routing organizes execution; the cluster still needs appropriate network and workload security policy.

Promote a release with evidence

Before sending normal traffic, run a fixed evaluation set against the candidate image. Include malformed requests, denied tools, provider timeouts, and interrupted sessions—not just successful prompts.

Record the image digest, component revision, evaluation dataset, and results with Polyaxon tracking. Decide which failures block release in your deployment workflow; logging a metric alone does not enforce that decision.

Plan rollback and draining together. Stop accepting new work before retiring an old service, and reconcile requests that were active during the transition. This makes the service a dependable part of the platform rather than a long-running prototype.