Mount model artifacts with Kubernetes image volumes

A serving team changes its model more often than its inference runtime. Baking both into one container image ties two release cycles together. Downloading weights during every startup separates them, but introduces a bootstrap step that must handle credentials, failures, and partial files.

Kubernetes image volumes offer another option: mount the contents of an OCI image or artifact as a read-only volume. The feature is stable in Kubernetes 1.36 and enabled by default, but it requires support from the container runtime. Image volume task guide.

For a model server, this can make the runtime image and the model package two independently versioned inputs to the same deployment.

Define the artifact boundary

Suppose a release consists of an inference runtime, model weights, tokenizer files, and a serving configuration. Keep the files that must agree in one model package. Record the runtime image digest and model artifact digest separately in the release manifest.

This separation helps answer a practical rollback question: did the team change the execution software, the model files, or both? A familiar tag such as production does not answer that question if its target can move.

An image volume reference can use a digest. Its contents are resolved when the Pod starts; changing a tag in the registry does not update files inside an existing mount. Pull policy, registry authentication, and runtime support affect whether startup succeeds. See the image volume reference.

Mount the model and keep writes elsewhere

This Pod-spec fragment shows the intended separation. It assumes a compatible Linux runtime, an accessible OCI model package, and an application image that reads its model from /models/release. Replace both digest placeholders with real digests and supply the application's remaining configuration.

spec:
  containers:
    - name: inference
      image: registry.example.com/ml/server@sha256:REPLACE_RUNTIME_DIGEST
      volumeMounts:
        - name: model-release
          mountPath: /models/release
          readOnly: true
        - name: runtime-cache
          mountPath: /var/cache/model-server
      resources:
        requests:
          cpu: "2"
          memory: "8Gi"
        limits:
          memory: "8Gi"
  volumes:
    - name: model-release
      image:
        reference: registry.example.com/ml/model@sha256:REPLACE_MODEL_DIGEST
        pullPolicy: IfNotPresent
    - name: runtime-cache
      emptyDir:
        sizeLimit: "10Gi"

The resource values are illustrative, and no GPU allocation is included. Add the resources required by your actual server. Configure its cache path explicitly; mounting a directory does not cause a library to start using it.

Model conversion, kernel compilation, download caches, and generated indexes may need writable storage. Put those writes in an appropriate separate volume. The emptyDir above is disposable scratch space, not a place for durable outputs, and its size limit does not replace node storage planning.

Also confirm the package's directory layout. A successful mount is not enough if the server expects config.json at a different path or the tokenizer belongs to another model revision.

Choose between three delivery patterns

An init container can prepare a shared directory before the application runs. That remains useful when preparation requires transformation rather than just exposing immutable files. A persistent volume remains useful when the storage contract matters more than the packaging format.

Image volumes do not make a large model free to distribute. A cold node still needs to obtain the artifact. Registry bandwidth, node storage, and the number of replicas starting together can dominate rollout time. Measure cold and warm starts separately instead of assuming that reuse of OCI infrastructure guarantees a faster deployment.

Diagnose startup before changing the server

Follow a failed rollout in order: can the node resolve and authenticate to the registry, can the runtime obtain and mount the artifact, and can the application read the expected files? Pod events help separate image-volume preparation from an application failure after startup.

For private registries, review the supported credential sources, including the Pod's imagePullSecrets. Permissions used by an application to access object storage do not automatically authenticate the node to an OCI registry.

Once the server starts, use a readiness check that reflects its ability to serve requests. A mounted model directory is an input to readiness, not evidence that model loading or warmup has finished. The model-server startup probe guide covers that next boundary.

Preserve both versions in Polyaxon

Polyaxon gives the packaging decision a useful experiment record. For each inference benchmark or evaluation, record the runtime-image digest and model-artifact digest as run metadata, then retain the measured startup time, latency, and evaluation results with that run. Changing only the model package becomes a comparison you can inspect rather than an ambiguous image update.

Use Polyaxon's artifact connections for persistent outputs and shared datasets, and artifact logging to retain reports or references to externally stored model packages. The read-only model mount and the writable results store serve different purposes. A notebook exploring a model, an evaluation job, and an inference benchmark can all record the same immutable model reference while producing their own outputs.

For image-volume delivery itself, verify that the deployed Polyaxon schema and workload controller preserve the Kubernetes field alongside the supported volume configuration. Try one immutable model release, check its file layout and writable paths, then compare cold starts and rollback using the recorded digests.