Optimize LLM inference with repeatable benchmarks
Compare inference optimizations against a fixed workload, latency limits, and quality checks, with benchmark configurations and results tracked in Polyaxon.
LLMOpsInferenceGuides
All blog articles
Practical guides to building, running, and improving ML and AI in production.
All articles
Page 4 of 22
Combine run filters with AND and OR
Use OR to combine metric thresholds, negated conditions, and independent groups of filters in Polyaxon queries.
ProductCliGuides
Control notebook and inference service lifetimes
Reuse Polyaxon's termination specification to bound notebook and inference service lifetimes, stop idle services, and distinguish inactivity from ongoing work.
PolyaxonSchedulingNotebooks
Automate multiple runs with async Python clients
Monitor several Polyaxon runs concurrently with async Python clients, retrieve recent logs, and manage concurrency, timeouts, and client cleanup.
ProductPythonGuides
Tune LLM hyperparameters within a compute budget
Design a small LoRA fine-tuning sweep, control its execution in Polyaxon, and compare validation quality, resource use, and repeatability.
Hyperparameter TuningLLMOpsGuides
Control Kubernetes Job retries with pod failure policies
Stop retrying permanent errors, preserve the retry budget for marked disruptions, and inspect Kubernetes Job failure decisions with a concrete example.
KubernetesSchedulingTroubleshooting
Write shorter Polyaxonfiles
Use direct parameter values, let input and output defaults imply optionality, and simplify workload definitions with Polyaxon 2.18.
ProductCliGuides
Connect LLM production traces to evaluation runs
Keep release, dataset, evaluator, and sampling context connected as you investigate LLM failures and compare fixes with Polyaxon.
LLMOpsObservabilityEvaluation
Mount model artifacts with Kubernetes image volumes
Separate immutable model files from the serving image with Kubernetes image volumes, and choose when an OCI artifact fits better than a download or PVC.
KubernetesStorageInference
Route ML jobs to regional infrastructure with Polyaxon
Configure regional execution with Polyaxon agents, queues, and storage connections, then measure data-transfer costs without assuming same-region traffic is free.
InfrastructureSchedulingGuides
Unify hybrid AI platform operations
Operate AI workloads across clusters and clouds with central intent, local execution, placement policy, consistent identity, connected evidence, and failure-aware control.
Hybrid CloudInfrastructureMLOps
Request GPUs by capability with Kubernetes DRA
Use Dynamic Resource Allocation to describe the accelerator a workload needs, understand device claims, and plan the integration with Polyaxon.
KubernetesGPUScheduling
Upload multiple files and folders with Polyaxon CLI
Upload code, configuration, and small datasets in one Polyaxon CLI command, then save the same path mappings in a declarative mount section.
ProductCliGuides
Reduce the cost of self-hosted ML workloads
Reduce self-hosted ML costs with outcome-based accounting, right-sized resources, elastic capacity, interruption-ready workloads, local data paths, and deliberate retention.
InfrastructureKubernetesScheduling
Design multi-tenant AI agent infrastructure on Kubernetes
Separate identity, network access, data, queues, budgets, and evidence when multiple teams or customers run AI agents on shared Kubernetes infrastructure.
AI AgentsKubernetesSecurity
Restart containers selectively with Kubernetes restart rules
Use exit-code-based container restart rules, observe what stays within the Pod, and distinguish local recovery from retrying a complete ML operation.
KubernetesReliabilityWorkloads
Sandbox AI agents on Kubernetes
Design isolated Kubernetes execution for AI agents with a threat model, stronger runtimes, scoped identity, controlled egress, bounded storage, and auditable cleanup.
AI AgentsKubernetesSecurity
What is sovereign AI? Control across the AI lifecycle
Define sovereign AI as control over data, models, compute, operations, providers, and evidence, then implement it with Kubernetes.
InfrastructureKubernetesSecurity
Fine-tune Mistral 7B with LoRA on Kubernetes
Plan a Mistral 7B LoRA fine-tuning workflow on Kubernetes with versioned data, GPU scheduling, Polyaxon tracking, evaluation, and adapter packaging.
LLMOpsKubernetesGuides
Retry failed evaluation shards with Indexed Jobs
Give evaluation shards stable indexes and independent retry budgets with Kubernetes Indexed Jobs, while keeping incomplete results visible.
KubernetesEvaluationPipelines
Run Promptfoo evaluations on Kubernetes with Polyaxon
Package a Promptfoo suite as a Polyaxon job, export evaluation reports, track artifacts, and verify that failed checks fail the workload.
PromptfooEvaluationKubernetes
What are your ML jobs connecting to?
Trace image pulls, Git clones, S3 and GCS access, Hugging Face downloads, and artifact uploads across the lifecycle of Kubernetes jobs and sandboxes.
KubernetesObservabilityInfrastructure
Pods, Jobs, or Services for AI agents?
Choose the Kubernetes execution shape for an AI agent by separating logical task identity from Pods, Jobs, Services, workflows, and sandbox lifecycles.
AI AgentsKubernetesGuides
Orchestrate parallel coding agents with Polyaxon
Run parallel coding agents in separate Polyaxon workspaces, collect candidate patches, evaluate independently, and merge only reviewed changes.
AI AgentsSandboxesCollaboration