Pull private images with Polyaxon workload service accounts

A training image can live in a private registry without putting registry credentials in the training command. Kubernetes can use an image-pull Secret attached to the workload's ServiceAccount, and Polyaxon can select that ServiceAccount when it creates the operation's Pod. The credential is used for the image pull; the training process does not need to read it.

This is useful when a team runs several approved private images in the same compute namespace. An administrator owns the registry credential and Kubernetes ServiceAccount, while the Polyaxon component names the identity it needs. Polyaxon 2.10 added agent support for image-pull secrets on service accounts. The per-operation serviceAccountName field is documented in the custom ServiceAccount guide.

Prepare the identity in the workload namespace

First, have your cluster administrator provision a Secret of type kubernetes.io/dockerconfigjson for the registry. The Secret must be in the same namespace as the Pod that Polyaxon will create. Use your organization's credential-management process; do not paste a registry password into a checked-in manifest or a shared terminal transcript. Kubernetes's private-registry guide explains the Secret format and namespace rule.

Suppose the agent schedules this job in ml-team and the administrator has created a Secret named registry-pull there. Define a dedicated Kubernetes ServiceAccount in that namespace:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: ml-image-puller
  namespace: ml-team
imagePullSecrets:
  - name: registry-pull

The imagePullSecrets entry is a reference to the Secret, not a copy of its credential value. Kubernetes adds the ServiceAccount's image-pull references to newly created Pods that select it. A Kubernetes ServiceAccount also governs Pod identity for Kubernetes API access; keep any RBAC permissions narrow. It is different from a Polyaxon API service account used to authenticate a person or automation to the Polyaxon control plane.

Select it in a Polyaxon job

Save a component such as private-image-training.yaml:

version: 1.1
kind: component
name: private-image-training
run:
  kind: job
  environment:
    serviceAccountName: ml-image-puller
  container:
    image: registry.example.com/ml/train:approved
    command: ["python", "-m", "train"]

Replace the registry address and image with a real, accessible training image containing the train module. Pin its immutable digest for a production run so that every worker resolves the same build. Submit this component to a project and queue whose agent creates Pods in ml-team:

polyaxon run -f private-image-training.yaml

The name under run.environment.serviceAccountName selects the Kubernetes ServiceAccount for this operation. It does not create the account or the registry Secret. If the run also uses Polyaxon auxiliary containers, review the ServiceAccount specification and grant only the permissions those helpers actually require.

Confirm the new Pod got the right pull reference

An administrator with access to the compute namespace can inspect names without revealing registry credentials:

kubectl get serviceaccount ml-image-puller -n ml-team \
  -o jsonpath='{.imagePullSecrets[*].name}{"\n"}'

kubectl get pod POD_NAME -n ml-team \
  -o jsonpath='{.spec.serviceAccountName}{"\n"}{.spec.imagePullSecrets[*].name}{"\n"}'

kubectl describe pod POD_NAME -n ml-team

Replace POD_NAME with the Pod created for this operation. Expect the first two commands to show registry-pull and ml-image-puller in their respective fields. Then read the Pod events: a successful pull and container start are stronger evidence than the presence of a Secret reference alone. If you change the ServiceAccount, check a new operation; the Kubernetes guide describes injection for newly created Pods.

If the Pod reports ImagePullBackOff, check the image reference, namespace, Secret type and name, registry permissions, and Pod events. A credential for one repository may not authorize another. Our ImagePullBackOff guide walks through those failure signals. Avoid printing or decoding the Secret during routine diagnosis.

Choose the appropriate scope

The ServiceAccount route is useful when several operations in one namespace share an approved registry policy. For a one-off operation, Polyaxon also documents run.environment.imagePullSecrets as a direct list of Secret names. In either case, the Secret must exist where the Pod runs. A pull credential does not grant the application permission to push images, access a cloud bucket, or call the Kubernetes API.

Keep the component's image reference, the selected agent namespace, the ServiceAccount name, and the Secret owner in your runbook. That small chain makes a failed pull diagnosable without embedding credentials in the Polyaxonfile.