Fine-tune Mistral 7B with LoRA on Kubernetes
Plan a Mistral 7B LoRA fine-tuning workflow on Kubernetes with versioned data, GPU scheduling, Polyaxon tracking, evaluation, and adapter packaging.
LLMOpsKubernetesGuides
Kubernetes
Learn Kubernetes fundamentals, troubleshoot GPU workloads, and monitor ML infrastructure with practical guides for data scientists and platform engineers.
All articles
Page 2 of 8
Retry failed evaluation shards with Indexed Jobs
Give evaluation shards stable indexes and independent retry budgets with Kubernetes Indexed Jobs, while keeping incomplete results visible.
KubernetesEvaluationPipelines
Run Promptfoo evaluations on Kubernetes with Polyaxon
Package a Promptfoo suite as a Polyaxon job, export evaluation reports, track artifacts, and verify that failed checks fail the workload.
PromptfooEvaluationKubernetes
What are your ML jobs connecting to?
Trace image pulls, Git clones, S3 and GCS access, Hugging Face downloads, and artifact uploads across the lifecycle of Kubernetes jobs and sandboxes.
KubernetesObservabilityInfrastructure
Pods, Jobs, or Services for AI agents?
Choose the Kubernetes execution shape for an AI agent by separating logical task identity from Pods, Jobs, Services, workflows, and sandbox lifecycles.
AI AgentsKubernetesGuides
Restore notebook workspaces from volume snapshots
Create a CSI snapshot of a notebook workspace, restore it into a separate PVC, and verify files and application state before switching workspaces.
KubernetesStorageNotebooks
Kubernetes for AI agents: A platform engineering guide
Design a Kubernetes platform for AI agents with clear workload boundaries, durable state, scoped access, resource controls, and end-to-end evidence.
AI AgentsKubernetesInfrastructure
Hold Pods until they are ready for scheduling
Use Kubernetes scheduling gates to wait for external prerequisites before placement, with a controlled example and clear ownership of gate removal.
KubernetesSchedulingPipelines
Design reliable self-hosted ML infrastructure
Build self-hosted ML infrastructure around explicit failure domains, durable queues and artifacts, eligible failover, actionable telemetry, and tested recovery.
InfrastructureKubernetesMonitoring
Autoscale inference services using workload metrics
Use custom metrics with Kubernetes HPA for independent inference replicas, account for model startup, and distinguish desired replicas from usable capacity.
KubernetesInferenceMonitoring
Multi-cluster GPU orchestration
Design multi-cluster GPU orchestration around workload eligibility, data locality, queue routing, recovery, and clear dispatch ownership.
OrchestrationKubernetes
Run batch LLM evaluations on Kubernetes
Design reliable batch LLM evaluations with stable shards, bounded concurrency, GPU-aware placement, resumable results, and complete aggregation.
EvaluationKubernetesLLMOps
Operating long-running AI agents
Choose jobs, services, queues, and capacity controls for agent workloads using task duration, dependency limits, resume latency, and execution cost.
AI AgentsOrchestrationKubernetes
Roll out a new model when every GPU is occupied
Plan GPU inference rollouts around surge capacity, temporary reduced availability, model warmup, and request draining with a worked Deployment scenario.
KubernetesInferenceDeployment
Share CPU and memory across containers in a Pod
Use Pod-level resource budgets for cooperating containers, understand the remaining isolation boundaries, and evaluate the fit for ML services.
KubernetesResourcesInference
Place ML workloads close to their data
Design region-aware ML execution around dataset, registry, model, cache, artifact, and service locality without confusing proximity with data residency.
InfrastructureKubernetesGuides
Trace a failed training run to an unhealthy GPU
Connect Kubernetes DRA allocations, device health, and application failures without confusing missing telemetry with healthy hardware.
KubernetesGPUTroubleshooting
Docker build caching for ML workloads on Kubernetes
Reduce container build and startup time for ML jobs with reusable dependency layers, persistent build caches, and deliberate image and model caching.
DockerKubernetesInfrastructure
Understand Kubernetes workload-aware preemption
Understand how Kubernetes preempts capacity for PodGroups, choose disruption behavior for training and evaluation, and keep recovery separate from priority.
KubernetesSchedulingDistributed Training
Gang scheduling for distributed training
Understand how gang scheduling prevents partial distributed jobs from holding GPUs, how minimum membership works, and what Polyaxon supports.
SchedulingKubernetes
Design open infrastructure for portable AI workloads
Keep AI workloads portable with explicit execution contracts, open packaging and telemetry, hardware abstraction, data boundaries, and tested migration paths.
InfrastructureKubernetesMLOps
Run ML workloads on your existing Kubernetes cluster
Plan a Polyaxon deployment on an existing Kubernetes cluster, covering GPUs, storage, identity, networking, workload ownership, and recovery.
KubernetesInfrastructureGuides
GPU cluster scheduling tools compared
Compare Kueue, KAI Scheduler, Volcano, Coscheduling, and Slurm Bridge by responsibility, workload fit, and Polyaxon integration path.
SchedulingKubernetes
Build a conversational assistant on Kubernetes
Design a production conversational assistant with grounded retrieval, controlled model access, durable conversation state, evaluation, security, and Kubernetes operations.
LLMOpsKubernetesAI Agents