Kubernetes multi-tenancy for ML platforms

Multi-tenancy lets teams, projects, environments, or customers share Kubernetes infrastructure. For ML platforms, sharing can improve accelerator utilization and reduce the number of clusters to operate. It also creates competition for expensive resources and expands the consequences of weak identity or isolation policy.

Begin by defining who the tenants are and how much they trust one another. A namespace layout is not a threat model.

Choose the tenancy model

The Kubernetes multi-tenancy documentation distinguishes common organizational patterns and isolation mechanisms. A practical design starts with two broad cases:

• Trusted-team or soft multi-tenancy: tenants belong to the same organization and policy primarily prevents accidental interference.
• Untrusted or hard multi-tenancy: tenants may be hostile or subject to strict contractual or regulatory isolation.

A shared cluster can support strong controls, but it still shares a control plane, kernel-bearing worker nodes, and cluster-wide components. For some adversarial, sovereignty, or compliance requirements, separate clusters or accounts provide a clearer boundary.

Build identity before permissions

Map human users, automation, and workloads to distinct identities. Avoid shared administrator credentials and the default service account as a universal workload identity.

Use Kubernetes RBAC to grant verbs on specific resources and namespaces. Use Polyaxon RBAC for platform organizations, projects, and resources. Review both layers so a user cannot bypass an intended boundary through a lower-level credential.

Service accounts should be scoped to the workload's task. Disable automatic token mounting where the Pod does not need the Kubernetes API, and rotate external credentials independently of images and manifests.

Treat namespaces as an organizing boundary

Namespaces scope names and many policy objects, but they do not automatically isolate network traffic, nodes, cluster-scoped resources, or every controller. Apply a consistent tenant package:

• RBAC roles and bindings;
• ResourceQuota and LimitRange;
• default-deny ingress and egress policy with required exceptions;
• Pod Security Admission settings;
• approved service accounts and connections;
• storage classes and PVC policy;
• workload labels for ownership and cost;
• observability views and retention.

Automate namespace creation so tenants do not receive partially configured environments.

Separate capacity policy from Pod placement

ResourceQuota limits aggregate requested resources in a namespace; it does not create GPU capacity or guarantee fair scheduling time. Taints protect specialized nodes; tolerations do not reserve them. Priority can improve access for important work but can also displace other tenants.

Use queues and concurrency policy for shared ML capacity. Polyaxon queues can express priority, concurrency, quota, and routing across namespaces or clusters. Combine them with accurate resource requests and controlled scheduling presets.

Publish which policy decides admission, which decides placement, and how waiting work is ordered. Users should not need to infer fairness from a Pending Pod.

Isolate network and data

Apply NetworkPolicy with a plugin that enforces it. Begin from required flows: DNS, artifact storage, registries, Git, identity, telemetry, and explicitly approved tenant-to-tenant services. A namespace boundary alone does not block Pod traffic.

Separate credentials and storage prefixes by tenant. A shared bucket with one cluster-wide key defeats namespace isolation. Configure Polyaxon connections with tenant-appropriate credentials and expose only required connections to each project or operation.

Persistent volumes and snapshots require their own ownership and deletion policy. Confirm that retained data cannot be mounted, restored, or inspected by another tenant through broader infrastructure permissions.

Protect shared components

DNS, ingress, metrics, logging, admission controllers, device plugins, schedulers, and the API server are shared failure domains. Enforce limits on API requests, events, log volume, metric cardinality, object count, and controller behavior.

Avoid letting tenants install arbitrary cluster-wide operators or privileged DaemonSets. Admission policy should reject unsafe host mounts, privilege, host networking, and unapproved capabilities according to the tenancy model.

Design tenant-aware observability

Teams need enough evidence to diagnose their workloads without listing another tenant's object names, labels, logs, metrics, or costs. Scope dashboards and queries, protect raw telemetry backends, and audit administrative access.

Platform operators still need a cluster-wide view for capacity and incidents. Keep that broader access explicit, reviewed, and logged. Correlate infrastructure evidence with Polyaxon project and run metadata without placing sensitive run identifiers into unbounded metric labels.

Test isolation and noisy-neighbor behavior

Validate policy with real scenarios:

• one tenant exhausts its CPU, memory, GPU, PVC, object, or log quota;
• a workload attempts cross-namespace access;
• a user tries to bind a broader role;
• a large job competes with an interactive service;
• a tenant emits high-cardinality metrics or API requests;
• a node or cluster fails while several tenants have active work.

The goal is graceful containment: one tenant reaches a documented limit, receives an explainable status, and does not make unrelated workloads unavailable.

Multi-tenancy succeeds when identity, policy, capacity, data, and evidence tell the same story. If the required isolation cannot be demonstrated under failure and adversarial conditions, use a stronger boundary rather than relying on the word “namespace.”