Polyaxon v3 is coming →
TRL

Fine-tune with TRL on Polyaxon

TRL owns the training loop; Accelerate, FSDP, or DeepSpeed can own process distribution. Polyaxon packages that stack as a repeatable operation and keeps its configuration, logs, lineage, and artifacts together.

Outcome

A GPU-backed TRL component that can be promoted from a single-node SFT run to a distributed recipe without changing the surrounding operating model.

Start the guideRead the TRL CLI reference

Execution responsibilities

Polyaxon resolves

The component pins the image, inputs, secrets, resources, queue, and output contract.

Kubernetes places

The cluster schedules the requested accelerator topology and enforces namespace policy.

The framework trains

The training stack owns optimization while Polyaxon records status, logs, metadata, and artifacts.

Prerequisites

  • A Kubernetes GPU node compatible with the CUDA stack in the pinned TRL image.
  • A Hugging Face token connection only when the selected model or dataset is gated.
  • A Polyaxon artifacts store configured for checkpoints and the trained model.

Run and validate

  1. 1

    Define the TRL component

    Use TRL's published image and CLI, and expose the model and dataset as component inputs.

    version: 1.1kind: componentname: trl-sftinputs:- name: model  type: str  value: Qwen/Qwen2.5-0.5B- name: dataset  type: str  value: trl-lib/Capybararun:  kind: job  container:    image: huggingface/trl:1.9.2    command: ["trl", "sft"]    args:    - "--model_name_or_path"    - "{{ model }}"    - "--dataset_name"    - "{{ dataset }}"    - "--output_dir"    - "{{ globals.run_outputs_path }}/qwen2.5-0.5b-sft"    resources:      limits:        nvidia.com/gpu: "1"

    This uses Hugging Face's published TRL image and the public model and dataset from its CLI example. Pin the image digest and model and dataset revisions before production.

  2. 2

    Submit the operation

    Run the component through the target Polyaxon project, queue, preset, and approval path.

    polyaxon run -f trl.yaml
  3. 3

    Compare the resolved training run

    Confirm the image, revisions, parameters, accelerator allocation, logs, checkpoints, and evaluation metrics before scaling out.

    polyaxon ops dashboard

Production checklist

Pin the container image, framework versions, model revision, dataset revision, and training configuration.
Resolve registry, dataset, model, and tracking credentials through scoped Polyaxon connections.
Persist checkpoints and final artifacts outside the pod, and define resume behavior before a long run.
Set requests, limits, node selectors, tolerations, queue policy, timeout, and retry ownership explicitly.
For multi-node training, choose one distribution owner—such as the PyTorchJob operator or a framework launcher—and validate rendezvous and checkpoint recovery.

Troubleshooting

The pod remains Pending

Inspect quota, queue capacity, node selectors, taints, topology, and the requested accelerator count.

The process runs out of memory

Check model precision, sequence length, batch size, checkpointing, optimizer state, sharding, and actual device memory.

The run cannot read or save

Verify the referenced connection, secret scope, mounted path, object-store permissions, and available storage.

Sources

Official platform, library, model, and Polyaxon references used by this guide.

Continue

Compare AI platformsExplore MLOps