Mount model artifacts with Kubernetes image volumes
Separate immutable model files from the serving image with Kubernetes image volumes, and choose when an OCI artifact fits better than a download or PVC.
KubernetesStorageInference
Kubernetes
Learn Kubernetes fundamentals, troubleshoot GPU workloads, and monitor ML infrastructure with practical guides for data scientists and platform engineers.
All articles
Page 2 of 8
Request GPUs by capability with Kubernetes DRA
Use Dynamic Resource Allocation to describe the accelerator a workload needs, understand device claims, and plan the integration with Polyaxon.
KubernetesGPUScheduling
Reduce the cost of self-hosted ML workloads
Reduce self-hosted ML costs with outcome-based accounting, right-sized resources, elastic capacity, interruption-ready workloads, local data paths, and deliberate retention.
InfrastructureKubernetesScheduling
Design multi-tenant AI agent infrastructure on Kubernetes
Separate identity, network access, data, queues, budgets, and evidence when multiple teams or customers run AI agents on shared Kubernetes infrastructure.
AI AgentsKubernetesSecurity
Restart containers selectively with Kubernetes restart rules
Use exit-code-based container restart rules, observe what stays within the Pod, and distinguish local recovery from retrying a complete ML operation.
KubernetesReliabilityWorkloads
Sandbox AI agents on Kubernetes
Design isolated Kubernetes execution for AI agents with a threat model, stronger runtimes, scoped identity, controlled egress, bounded storage, and auditable cleanup.
AI AgentsKubernetesSecurity
What is sovereign AI? Control across the AI lifecycle
Define sovereign AI as control over data, models, compute, operations, providers, and evidence, then implement it with Kubernetes.
InfrastructureKubernetesSecurity
Fine-tune Mistral 7B with LoRA on Kubernetes
Plan a Mistral 7B LoRA fine-tuning workflow on Kubernetes with versioned data, GPU scheduling, Polyaxon tracking, evaluation, and adapter packaging.
LLMOpsKubernetesGuides
Retry failed evaluation shards with Indexed Jobs
Give evaluation shards stable indexes and independent retry budgets with Kubernetes Indexed Jobs, while keeping incomplete results visible.
KubernetesEvaluationPipelines
Run Promptfoo evaluations on Kubernetes with Polyaxon
Package a Promptfoo suite as a Polyaxon job, export evaluation reports, track artifacts, and verify that failed checks fail the workload.
PromptfooEvaluationKubernetes
What are your ML jobs connecting to?
Trace image pulls, Git clones, S3 and GCS access, Hugging Face downloads, and artifact uploads across the lifecycle of Kubernetes jobs and sandboxes.
KubernetesObservabilityInfrastructure
Pods, Jobs, or Services for AI agents?
Choose the Kubernetes execution shape for an AI agent by separating logical task identity from Pods, Jobs, Services, workflows, and sandbox lifecycles.
AI AgentsKubernetesGuides
Restore notebook workspaces from volume snapshots
Create a CSI snapshot of a notebook workspace, restore it into a separate PVC, and verify files and application state before switching workspaces.
KubernetesStorageNotebooks
Kubernetes for AI agents: A platform engineering guide
Design a Kubernetes platform for AI agents with clear workload boundaries, durable state, scoped access, resource controls, and end-to-end evidence.
AI AgentsKubernetesInfrastructure
Hold Pods until they are ready for scheduling
Use Kubernetes scheduling gates to wait for external prerequisites before placement, with a controlled example and clear ownership of gate removal.
KubernetesSchedulingPipelines
Design reliable self-hosted ML infrastructure
Build self-hosted ML infrastructure around explicit failure domains, durable queues and artifacts, eligible failover, actionable telemetry, and tested recovery.
InfrastructureKubernetesMonitoring
Autoscale inference services using workload metrics
Use custom metrics with Kubernetes HPA for independent inference replicas, account for model startup, and distinguish desired replicas from usable capacity.
KubernetesInferenceMonitoring
Multi-cluster GPU orchestration
Design multi-cluster GPU orchestration around workload eligibility, data locality, queue routing, recovery, and clear dispatch ownership.
OrchestrationKubernetes
Run batch LLM evaluations on Kubernetes
Design reliable batch LLM evaluations with stable shards, bounded concurrency, GPU-aware placement, resumable results, and complete aggregation.
EvaluationKubernetesLLMOps
Operating long-running AI agents
Choose jobs, services, queues, and capacity controls for agent workloads using task duration, dependency limits, resume latency, and execution cost.
AI AgentsOrchestrationKubernetes
Roll out a new model when every GPU is occupied
Plan GPU inference rollouts around surge capacity, temporary reduced availability, model warmup, and request draining with a worked Deployment scenario.
KubernetesInferenceDeployment
Share CPU and memory across containers in a Pod
Use Pod-level resource budgets for cooperating containers, understand the remaining isolation boundaries, and evaluate the fit for ML services.
KubernetesResourcesInference
Place ML workloads close to their data
Design region-aware ML execution around dataset, registry, model, cache, artifact, and service locality without confusing proximity with data residency.
InfrastructureKubernetesGuides
Trace a failed training run to an unhealthy GPU
Connect Kubernetes DRA allocations, device health, and application failures without confusing missing telemetry with healthy hardware.
KubernetesGPUTroubleshooting