Design multi-tenant AI agent infrastructure on Kubernetes

Two teams can safely share a Kubernetes cluster only when sharing does not imply shared authority. For AI agents, that authority extends beyond cluster resources: an agent may use model credentials, retrieve protected data, invoke internal tools, create child tasks, and retain state between interactions.

Multi-tenancy therefore needs several coordinated boundaries. A namespace is useful for organization and policy selection, but it is not, by itself, a complete security, cost, or data-isolation model.

Define the tenant before the namespace

Decide which entity requires separation: a customer, business unit, project, environment, user, or workload trust class. The answer determines which resources may be shared and which failures must not cross the boundary.

Document the isolation objectives for each tenant relationship:

The Kubernetes multi-tenancy documentation distinguishes softer sharing between trusted teams from harder multi-tenancy for untrusted tenants. Stronger requirements may justify dedicated nodes, runtime classes, clusters, or cloud accounts rather than adding more policies to one shared namespace model.

Use namespaces as policy attachment points

Give tenants or projects clear namespace ownership and avoid running unrelated workloads in a common default namespace. Namespaces help scope names, RBAC bindings, quotas, limit ranges, network policies, and admission rules.

Prevent tenants from creating or modifying the controls that isolate them. Restrict namespace creation, RoleBindings, webhook configuration, privileged workloads, host access, runtime selection, and other cluster-scoped capabilities. Review any operator that watches multiple namespaces; its service account may bridge otherwise separate boundaries.

Keep platform services in separately managed namespaces with narrow interfaces. A shared dispatcher, model gateway, artifact proxy, or telemetry collector should authenticate every request rather than trusting its network location.

Bind identity to tenant and workload

Use distinct ServiceAccounts for workload classes instead of one broad identity per namespace. Bind each to the minimum Kubernetes permissions it needs, and disable token mounting for containers that do not call the Kubernetes API.

Map external workload identity to the same tenant and environment. Separate credentials for model providers, artifact stores, repositories, data systems, and production tools. A shared secret with a tenant ID in the request body is not tenant isolation; the credential itself must be appropriately scoped or mediated by a policy-enforcing service.

When an agent acts on behalf of a user, record both identities. The workload identity explains which process made the call; the delegated subject explains whose authority the task was allowed to use. Bound delegation by action, resource, task, and expiration where the tool supports it.

Default-deny network paths

Apply ingress and egress policy for every tenant workload class. Allow traffic to named platform services through stable selectors and ports, and tightly control access to other namespaces. Explicitly address DNS, artifact storage, model endpoints, telemetry, and any proxy that can reach broader networks.

Kubernetes NetworkPolicy does not by itself understand user identity, tool permissions, or arbitrary domain allowlists. Combine network controls with service-level authentication and authorization. A tool gateway should still reject an action that the tenant is not permitted to perform, even if the agent can reach its address.

Collect network flow evidence where practical and connect it to workload and task identities. This helps distinguish an application timeout from a denied path and provides evidence for unexpected destinations.

Separate queues, quotas, and application budgets

Kubernetes ResourceQuota can limit aggregate resource consumption in a namespace. LimitRange and admission policies can require sensible per-workload requests and limits. These controls protect cluster capacity but do not determine which queued task should run next.

Use admission and queue policy to express tenant concurrency, priority, and fair access to scarce node pools or GPUs. Reserve capacity only where an SLO justifies it, and prevent one bursty tenant from occupying every eligible slot.

Add application budgets for hosted model calls, tokens, tool actions, storage, and child tasks. Preserve consumed budget across retries and resumed sessions. Infrastructure quota cannot prevent a low-CPU agent from generating a large external API bill.

Report queue delay, resource-hours, provider cost, retries, and accepted task outcomes by tenant. Avoid optimizing for allocation alone: a tenant can hold capacity without producing useful work, or produce expensive traffic while using little local compute.

Partition data and retained state

Use tenant-scoped storage prefixes, credentials, encryption keys, retention policies, and indexes where the risk requires them. Do not depend on application-supplied path strings to prevent traversal into another tenant's artifacts.

Treat caches as shared data stores. Prompt caches, model-response caches, package caches, embeddings, retrieval indexes, and warm workspaces can leak information if their keys and access controls omit tenant context. Decide which caches are safe to share and include all security-relevant inputs in the key.

Reset writable workspaces between unrelated tasks. If an agent session is intentionally persistent, associate it with one tenant and compatible application revision, then define how it expires and how its outputs are reviewed before reuse.

Keep audit evidence and application content separable. Operators may need status, timing, policy decisions, and resource usage without permission to read prompts, documents, or tool results. Apply redaction and field-level access instead of placing sensitive content into broadly visible Kubernetes labels or events.

Govern tools independently from cluster access

Tool calls can cross boundaries that Kubernetes cannot see. An agent with access to a generic browser, shell, database client, or repository token may convert a narrow network exception into broad authority.

Publish a tool catalog with owner, allowed actions, input schema, tenant rules, credential strategy, side-effect class, and audit requirements. Route sensitive actions through policy enforcement that validates the task and delegated user context. Require approval for high-impact operations and make the approval refer to the exact action revision.

Separate read and write capabilities. A research agent that can inspect deployment status does not automatically need permission to change it. Use synthetic or read-only environments for evaluations and red-team exercises unless the scenario explicitly requires controlled side effects.

Isolate untrusted execution more strongly

Tenant separation and code trust are different axes. Two teams inside one company may trust each other's container images but still require data and cost separation. Generated code from either team may require a stronger runtime boundary.

Select security context, sandboxed runtime, node pool, or cluster isolation according to both axes. The guide to sandboxing AI agents on Kubernetes covers runtime, filesystem, network, identity, and cleanup controls for generated code.

Warm pools need special care. A pre-provisioned environment can reduce startup latency, but the claim process must attach the correct tenant policy and ensure no writable state, credential, or network exception survives from a previous claimant.

Preserve an attributable evidence chain

Every task should connect the tenant, project, submitter, delegated subject, logical task, execution attempts, workload identity, policy revision, code and prompt versions, tool actions, retained artifacts, and evaluations.

Use separate views for platform health and tenant behavior, linked by stable IDs. Kubernetes logs and events explain placement, restart, and resource problems. Agent traces explain model, retrieval, tool, and decision paths. Neither view alone is enough to investigate a cross-tenant concern.

Define who may search each field and how long it is retained. Audit access to the evidence itself. A centralized observability system can become a data exfiltration path if every team can query every prompt or tool response.

Map the boundaries to Polyaxon

Use Polyaxon organizations, projects, RBAC, connections, presets, and queues according to the edition and deployment model you operate. Projects provide an application and collaboration boundary; Kubernetes namespaces and cluster policy provide an execution boundary. Align their ownership rather than assuming one automatically configures the other.

Attach tenant-appropriate connections to operations, and scope their underlying credentials. Apply common resource, scheduling, and security settings through managed presets. Commercial queues can express priority, concurrency, quota, and routing across namespaces or clusters.

Record outputs in approved artifact stores and retain execution metadata needed for cost allocation and investigation. Polyaxon's workload orchestration does not replace application-level tool authorization or model budgets; carry the tenant and task context through those systems explicitly.

For environments with harder isolation requirements, route workloads to dedicated namespaces, node pools, agents, or clusters as supported by the deployment. The Kubernetes platform guide for AI agents explains the broader responsibility split.

Review the boundary with adversarial cases

Test with synthetic tenants and credentials. Attempt cross-namespace reads, unauthorized tool actions, artifact path manipulation, cache-key collisions, quota exhaustion, child-task fan-out, access after cancellation, and reuse of a dirty warm environment.

Confirm that denial is visible and attributable without exposing another tenant's content. Then repeat the review when identity providers, network plugins, storage configuration, admission policy, queues, agent runtimes, or shared platform services change.

A multi-tenant agent platform is credible when a team can state what is shared, what is isolated, which system enforces each boundary, and which evidence proves that the boundary held.