Design open infrastructure for portable AI workloads

Open AI infrastructure is not infrastructure with no vendors or proprietary services. It is a platform whose critical workload, data, identity, telemetry, and artifact contracts remain inspectable and replaceable enough to preserve organizational choice.

Portability must be designed and tested. A container image alone does not make a workload portable when it depends on one scheduler, storage API, accelerator stack, model endpoint, identity system, or undocumented manual process.

Define the portable unit

Describe a workload through its code or image, command, inputs, outputs, resources, environment, identity, connections, scheduling constraints, lifecycle, and acceptance criteria. Separate these logical requirements from destination-specific implementation.

For example, ask for a high-memory accelerator class and required precision rather than a provider instance name. Reference an artifact connection rather than embedding one bucket URL and credential. Preserve the resolved hardware and destination with the run so abstraction does not remove accountability.

Version the contract. Portability depends on both sides agreeing on fields and behavior, not merely accepting the same file format.

Use open packaging and orchestration boundaries

OCI container images provide a common packaging boundary across compatible runtimes. Kubernetes provides APIs for workload reconciliation, identity attachment, networking, storage claims, and scheduling. Those standards reduce custom integration, but portable behavior still depends on supported versions and extensions.

Pin immutable images and keep build provenance. Avoid bundling destination credentials or writable state into the image. Store checkpoints and outputs through declared durable connections.

Use standard Kubernetes resources where they express the requirement. Add custom resources only with a clear owner, versioning policy, export path, and plan for clusters where the extension is unavailable.

Abstract hardware without hiding capabilities

Accelerators differ in memory, precision, interconnect, partitioning, drivers, and failure behavior. A lowest-common-denominator interface can waste those capabilities, while SKU-specific manifests create tight coupling.

Offer a small catalog of capability classes with documented guarantees. Kubernetes Dynamic Resource Allocation can represent device attributes and claims when supported by the deployed drivers. Conventional device plugins can remain part of the transition.

Keep escape hatches for measured workloads that need a specific device, but make the exception visible and reevaluate it when hardware changes.

Keep data movement explicit

Data is often less portable than compute. Record where datasets, indexes, checkpoints, models, caches, and outputs live; which identities may access them; and which regions or environments are eligible.

Separate logical artifact references from physical storage paths. Define transfer, caching, encryption, integrity, retention, and deletion behavior. A copied dataset or retrieval index creates a new governed asset.

Include data transfer time and egress cost in placement. A cluster with available compute is not an eligible destination if the data cannot move there safely or within the workload deadline.

Standardize telemetry and lineage

Use stable operation, attempt, project, artifact, deployment, and model identifiers across logs, metrics, traces, and lineage. OpenTelemetry provides common concepts for traces, metrics, and logs, but teams still need an organization-specific semantic contract.

Export data through documented interfaces and retain the raw context required to rebuild important dashboards. Avoid making incident response depend on one vendor-only correlation ID.

Portability does not mean copying every prompt or log to a central destination. Apply redaction, access, and retention according to the data class.

Preserve security across destinations

Translate identity and policy without weakening them. Each environment should provide workload identity, least-privilege data access, network isolation, secret delivery, admission policy, audit, and revocation.

Do not assume two clusters provide equivalent boundaries because both run Kubernetes. Compare managed control-plane settings, node trust, networking enforcement, storage encryption, operator access, and installed policy.

Express hard constraints before placement preferences. If a destination cannot satisfy a required control, the workload should remain ineligible.

Test migration as a normal exercise

Periodically run one representative training job, evaluation, service, and recovery flow on a secondary eligible environment. Verify images, data, credentials, accelerator allocation, telemetry, checkpoints, artifacts, and outcome equivalence.

Measure queue time, transfer time, performance, cost, and operator effort. Record every destination-specific change. That list is the real portability backlog.

Polyaxon components express repeatable workloads, connections separate logical dependencies, and agents can manage configured Kubernetes environments. Queues and presets can apply destination-specific routing and scheduling while tracking retains the resolved execution context.

Open infrastructure preserves leverage when it creates verified exit paths. The standard is not “could run elsewhere”; it is “has run elsewhere with the required behavior, security, evidence, and recovery.”