Scheduling articles

Resize CPU and memory without replacing Kubernetes Pods

Use Kubernetes in-place resource resizing for running workloads, inspect whether changes took effect, and understand the limits for Polyaxon services.

Native gang scheduling reaches beta in Kubernetes 1.37

Explore native gang scheduling in Kubernetes 1.37, its potential for Polyaxon training and sandboxes, and how it compares with KAI, Kueue, and Volcano.

Keep distributed training workers close together

Understand native topology-aware workload scheduling, combine rack locality with gang placement, and assess the tradeoff between waiting and communication.

Control notebook and inference service lifetimes

Reuse Polyaxon's termination specification to bound notebook and inference service lifetimes, stop idle services, and distinguish inactivity from ongoing work.

Control Kubernetes Job retries with pod failure policies

Stop retrying permanent errors, preserve the retry budget for marked disruptions, and inspect Kubernetes Job failure decisions with a concrete example.

Route ML jobs to regional infrastructure with Polyaxon

Configure regional execution with Polyaxon agents, queues, and storage connections, then measure data-transfer costs without assuming same-region traffic is free.

Request GPUs by capability with Kubernetes DRA

Use Dynamic Resource Allocation to describe the accelerator a workload needs, understand device claims, and plan the integration with Polyaxon.

Reduce the cost of self-hosted ML workloads

Reduce self-hosted ML costs with outcome-based accounting, right-sized resources, elastic capacity, interruption-ready workloads, local data paths, and deliberate retention.

Hold Pods until they are ready for scheduling

Use Kubernetes scheduling gates to wait for external prerequisites before placement, with a controlled example and clear ownership of gate removal.

Understand Kubernetes workload-aware preemption

Understand how Kubernetes preempts capacity for PodGroups, choose disruption behavior for training and evaluation, and keep recovery separate from priority.

Gang scheduling for distributed training

Understand how gang scheduling prevents partial distributed jobs from holding GPUs, how minimum membership works, and what Polyaxon supports.

GPU cluster scheduling tools compared

Compare Kueue, KAI Scheduler, Volcano, Coscheduling, and Slurm Bridge by responsibility, workload fit, and Polyaxon integration path.

How to improve GPU utilization

A practical guide to improving GPU utilization by diagnosing queue delays, input bottlenecks, resource fragmentation, sharing, and recovery overhead.

GPU utilization metrics: allocation, activity, and throughput

Learn which GPU utilization metrics explain capacity, device activity, memory pressure, and useful ML throughput—and how to avoid misleading averages.

What is GPU orchestration?

Understand how GPU orchestration connects workflows, queues, resource placement, and recovery across shared ML infrastructure.

Queue management for machine learning workloads

Why queue management matters for shared ML infrastructure and how Polyaxon handles priorities, concurrency, and workload scheduling.

Reduce AI agent startup latency with Polyaxon

Reduce Polyaxon agent startup latency by measuring queueing, image preparation, initialization, and readiness before tuning capacity or workspace reuse.

Kubernetes taints and tolerations for ML workloads

Keep ordinary Pods away from specialized nodes and combine tolerations with positive placement rules for GPU and interruptible ML capacity.

Kubernetes multi-tenancy for ML platforms

Design identity, isolation, quotas, queues, networking, storage, and observability for multiple ML teams sharing Kubernetes infrastructure.

Kubernetes CronJobs for ML automation

Schedule repeatable Kubernetes jobs with explicit time zones, concurrency, deadlines, history limits, idempotency, and observable outcomes.

Kubernetes nodes for ML platforms

Understand node components, conditions, capacity, labels, taints, failure behavior, and lifecycle management for Kubernetes ML clusters.

Diagnose waiting workloads with agent and queue history

Use Polyaxon queue and agent statistics with run timelines to distinguish policy limits, workload changes, and Kubernetes placement problems before adding capacity.

Right-size Kubernetes resources for ML workloads

Set CPU, memory, ephemeral-storage, and GPU resources from measured ML workload behavior while preserving scheduling efficiency and reliability.

Understand Kubernetes Pod evictions for ML

Distinguish node-pressure, API-initiated, preemption, and node-failure disruptions, then design ML workloads to recover safely.