Polyaxon v3 is coming →
NVIDIA Dynamo

Deploy NVIDIA Dynamo with Polyaxon

On Kubernetes, NVIDIA's Dynamo operator owns the inference graph and reconciles its frontend and worker components. Polyaxon can run the controlled release and evaluation stages around that native resource without pretending the graph is a single service pod.

Outcome

A versioned DynamoGraphDeployment release operation with an explicit service account, manifest image, and separate endpoint validation step.

Start the guideServing documentation

Service responsibilities

Polyaxon schedules

The service component fixes the image, model, port, accelerator request, connections, and health boundary.

The runtime serves

The inference engine loads the model and exposes its native or OpenAI-compatible API.

Clients validate

Smoke tests and evaluation operations exercise the endpoint while Polyaxon retains the deployment context.

Prerequisites

  • The NVIDIA GPU Operator and Dynamo Platform/operator installed in the target Kubernetes cluster.
  • A reviewed DynamoGraphDeployment manifest using the supported API version and a tested backend recipe.
  • A namespace-scoped service account allowed to manage only the required Dynamo resources.

Deploy and validate

  1. 1

    Release the Dynamo graph

    Package the reviewed manifest with a small release image. Polyaxon tracks who applied which revision; the Dynamo operator owns the resulting pods and services.

    version: 1.1kind: componentname: release-dynamo-graphrun:  kind: job  environment:    serviceAccountName: dynamo-release  container:    image: bitnami/kubectl:latest    command: ["kubectl", "apply", "-f", "/manifests/dynamo-graph.yaml"]

    The release operation does not make Polyaxon the Dynamo controller. Pin the kubectl image digest, and keep cleanup and rollback as explicit release or GitOps actions with the same scoped RBAC.

  2. 2

    Submit the operation

    Run the component through the target Polyaxon project, queue, preset, and approval path.

    polyaxon run -f dynamo-release.yaml
  3. 3

    Validate the operator-owned frontend

    Wait for the DynamoGraphDeployment to become ready, then run a separate Polyaxon evaluation or smoke-test job against its frontend service.

    kubectl wait --for=condition=Ready   dynamographdeployment/model-server --timeout=20m

Production checklist

Pin the runtime image and model revision; warm the model cache before measuring startup or latency.
Keep the service private by default and add authentication, TLS, rate limits, and network policy deliberately.
Define readiness, liveness, timeout, graceful shutdown, and rollback behavior before production traffic.
Measure latency distributions, throughput, quality, errors, and accelerator memory under representative load.
Version the DGD, backend image, parallelism, routing, storage, network, and autoscaling policy as one release artifact.
Use Dynamo's Kubernetes-native discovery rather than carrying local etcd/NATS assumptions into the cluster deployment.

Troubleshooting

The service never becomes ready

Inspect model download, credentials, disk space, GPU memory, runtime flags, port binding, and readiness behavior.

Requests fail through the URL

Verify the service port, rewrite-path setting, authentication header, ingress, network policy, and API path.

Latency degrades under load

Measure queueing, batch settings, context length, KV cache pressure, replica saturation, and storage or network contention.

Sources

Official platform, library, model, and Polyaxon references used by this guide.

Continue

Compare AI platformsExplore LLMOps