Changelogv3.0.0 · WIP
V3 · WIPv3.0.0 · WIP
WIP

Polyaxon v3 progress

Polyaxon v3 brings new ways to develop and debug agents, automate workloads, and manage infrastructure. It is fully backward compatible with v2. This page combines the product roadmap with the evolving v3 release notes.

Building toward v3

V3 is still in progress. We ship features incrementally, including in v2, so you can use improvements before the full release. A feature's maturity and its first shipped version are tracked separately below.

For example, you can already compare runs with synchronized charts in v2.14 and use sandbox execution, SSH access, and reconnectable shells in v2.16. The sandbox features remain in beta as they mature toward v3.

Feature progress

Select a status below to jump to its features. Shipped in links identify the first release containing a feature; alpha, beta, and development labels describe its maturity. Features without a shipped-version link should not be assumed available in a published release. Scope and maturity can change as development continues.

Completed

3

Completed work, including features already shipped in v2.

  • Richer run comparison

    Shipped in v2.14

    Compare dense run views with charts, graphs, synchronized cursors, shared selections, and parameter and metric tooltips.

    Color-coded heat fields comparing the final outputs of three runs in the runs table.
    Heat fields
    Six metric charts comparing three runs with synchronized cursors and tooltips at step 19.
    Metric charts
    Feature activation and loss landscape images displayed side by side for three runs.
    Image comparison
    Explore visualizations
  • Pass params without IO declarations

    Implemented for the upcoming 2.18 release. Undeclared params become context values; strictParams enables undeclared-name checks. Declared input and output validation still applies.

    Parameter reference
  • Omit the Polyaxonfile version

    Implemented for the upcoming 2.18 release. Components, operations, and compiled operations can omit version; existing versioned files remain valid. The field no longer selects a validation schema.

    Polyaxonfile specification

Beta

3

Features being refined as we work toward the final release.

  • Interactive debugging and sandboxes

    Shipped in v2.16

    Debug live workloads with sandbox command execution, file access, and interactive terminals.

    Explore sandboxes
  • Governed SSH access

    Shipped in v2.16

    Connect local terminals, IDEs, and file-transfer tools to running services through Polyaxon's authenticated SSH tunnel.

    Connect with SSH
  • Reconnectable shells with tmux

    Shipped in v2.16

    Enable the tmux plugin to disconnect and return to the same shell from the CLI or UI. Attach to tmux explicitly over SSH to preserve your shell across SSH disconnects. Sessions retain their working directory, variables, and running programs while the container and tmux process remain alive.

    Reconnect to your shell

Alpha

3

Early implementations that are still evolving.

  • Event-driven automation

    Connect hooks and platform events to evaluations, notifications, and downstream jobs when runs change.

  • Control plane integrations

    Manage built-in connections and integrations at the control plane level.

  • Incoming events

    Bring events from Git providers, webhooks, and external systems into Polyaxon workflows.

WIP

10

Features under development. Target releases are shown where planned.

  • Run agent-driven research campaigns

    Run bounded research campaigns that propose experiments, evaluate results, and retain evidence for each decision. Tune training configurations, agent prompts, or tool policies with explicit budgets and approvals. Track proposal branches, compare results against a baseline, and pause or resume across restarts. Configuration-only proposals come first, followed by isolated source experiments; numerical searches can delegate to native hyperparameter optimization.

  • Choose scoped credentials for each run

    Create personal access tokens with explicit permissions, service access, and expiration dates. Set a default token for new runs or select a specific token when submitting from the CLI or UI. Inspect token scopes and expiry, revoke access, and audit token creation and revocation. A staged rollout introduces scoped run credentials before retiring legacy token behavior as agents and CLIs are upgraded.

  • Connect agents (compute clusters) without installing Polyaxon

    Connect a Kubernetes cluster through cluster credentials or a linked cloud account, with Polyaxon hosting the agent and managing workloads remotely. No Polyaxon CRDs, operator, agent, or streams pods are installed in your cluster, so idle clusters need no resident Polyaxon pods or nodes reserved for them. Bring your own S3, GCS, or Azure artifact storage, and see connection, permission, and storage issues in agent status.

  • Managed artifact stores

    Manage artifact storage alongside the workloads that produce it.

  • LLM and agent gateway

    Access models through a gateway with fallbacks, load balancing, retries, custom guardrail hooks, and advanced evaluations.

  • Infrastructure observability

    Monitor agents, jobs, and clusters with built-in monitoring and observability.

  • Connect queues to external schedulers

    Explicitly bind Polyaxon queues to existing Kueue, Volcano, or KAI queues for native Trainer workloads. Follow runs from platform dispatch through cluster admission and scheduling, with pending reasons and scheduler messages visible in Polyaxon. View external queue usage and quotas where available, alongside Polyaxon's own concurrency, quota, and access controls.

  • Simpler Polyaxonfiles

    Targeting 2.18. Define jobs and services with less nesting, using container settings, commands, environment variables, and resources. Declare ports to select a service without an explicit kind field.

    Planned for Polyaxon 2.18. This example previews fine-tuning a language model with the TRL CLI.

    container:
      image: ghcr.io/astral-sh/uv:python3.12-trixie-slim
    
    env:
      MODEL: Qwen/Qwen2.5-0.5B
      DATASET: stanfordnlp/imdb
      GPUS_PER_NODE: "1"
    
    resources:
      cpu: "4..8"
      memory: "8Gi..16Gi"
      gpu: 1
    
    cmd:
      - uv pip install --system trl
      - |
        trl sft \
          --model_name_or_path "$MODEL" \
          --dataset_name "$DATASET" \
          --num_processes "$GPUS_PER_NODE"

    The uv image includes Python and uv; --system installs TRL into the container's Python environment. The workload needs a compatible NVIDIA GPU node and access to download the packages, model, and dataset. Keep GPUS_PER_NODE equal to the requested GPU count.

    Commands use cmd; run remains available for a complete native runtime specification. CPU and memory ranges define requests and limits. gpu: 1 requests exactly one NVIDIA GPU; GPU ranges are not supported.

    For an API, notebook, or inference endpoint, declare its ports at the top level:

    ports:
      - 8080

    Polyaxon infers a service without an explicit kind field. Your container command starts the server on that port. Without ports, the flat definition describes a job, as in the training example above.

  • Run without a Polyaxonfile

    Targeting 2.18. Define simple runs directly with CLI options, or start a sandbox with --sandbox.

    Planned for Polyaxon 2.18. These examples preview the upcoming CLI options.

    Fine-tune a language model on one NVIDIA GPU:

    polyaxon run \
      --image ghcr.io/astral-sh/uv:python3.12-trixie-slim \
      --env MODEL=Qwen/Qwen2.5-0.5B \
      --env DATASET=stanfordnlp/imdb \
      --env GPUS_PER_NODE=1 \
      --cmd 'uv pip install --system trl' \
      --cmd 'trl sft --model_name_or_path "$MODEL" --dataset_name "$DATASET" --num_processes "$GPUS_PER_NODE"' \
      --gpu 1 \
      --cpu 4..8 \
      --memory 8Gi..16Gi

    The image includes Python and uv. The first command installs TRL; the second starts training. Use a compatible NVIDIA GPU node with access to download the packages, model, and dataset. Keep GPUS_PER_NODE equal to --gpu.

    Expose a service on port 8080:

    polyaxon run \
      --image busybox:1.36 \
      --cmd 'httpd -f -p 8080' \
      --port 8080

    Start a sandbox with one NVIDIA GPU, without specifying a user port:

    polyaxon run --sandbox \
      --image python:3.12 \
      --gpu 1 \
      --cpu 4..8 \
      --memory 8Gi..16Gi

    CPU and memory ranges define requests and limits. Use --gpu 1 for an exact count of one NVIDIA GPU; GPU ranges are not supported. --port selects a service, while --sandbox selects a service with the sandbox plugin enabled.

  • Native hyperparameter optimization

    Improve batch search with native TPE v2 and Gaussian process Bayesian optimization, benchmarked against Random and Grid search. Reuse imported evaluations without rerunning them and allocate trial budgets asynchronously with ASHA and TPE+ASHA. Versioned policies keep active studies on their original behavior, with one optimization engine for model training and agent evaluation.

    Current hyperparameter tuning guide

Pending

9

On the roadmap; maturity and delivery timing are not yet confirmed.

  • Suspend and resume sandboxes

    Release compute when sandboxes are idle and wake them on access. Workspace files survive suspension; running processes and terminal sessions do not.

  • Warm pools(sandboxes)

    Reduce startup time for new runs with a pool of ready sandboxes. Used sandboxes never return to the pool.

  • Composable Python and TypeScript interfaces

    Specify logic, hardware, and execution through composable programmatic primitives.

  • Step-by-step agent debugging

    Test and debug agents with step-by-step execution, state inspection, and performance profiling in a sandbox environment.

  • LLM and agent observability

    Trace requests with OpenTelemetry-compatible observability. Capture inputs, outputs, and metadata for intermediate steps to identify bugs and unexpected behavior in LLM applications and agents.

  • Evaluations across development and production

    Evaluate and monitor GenAI applications throughout their lifecycle to measure quality, compare changes, and detect regressions.

  • Prompt management and optimization

    Manage and optimize prompts alongside application traces, evaluations, and model access in an integrated GenAI workflow.

  • Comments across Polyaxon

    Add comments to runs, models, artifacts, traces, spans, observations, and scores to share context and collaborate across teams.

  • Easier onboarding and faster dashboards

    Improve onboarding and core screens for new and nontechnical users, and improve UI performance for large dashboards.

V3.0.0

Draft release notes. These notes collect work toward v3.0.0, including improvements delivered incrementally in v2. They are not a list of newly available features. Use the feature tracker for maturity and shipped versions, and the changelog for published releases.

Core

  • New: Add new sandbox interface to allow running agent and AI-generated code in dynamically defined sandboxes.
  • Enhancement: Extract schedules to be a first-class concept in the system.

Infra

  • Enhancement: Move from Redis to Valkey and use the official chart.

UI

  • New: Add markdown, summary, lineage, and performance widgets.
  • New: Improve visualization of multiple runs in the compare dashboard.
  • Enhancement: Improve rendering and performance of charts.
  • Enhancement: Allow visualizing multiple images (index range) at the same time.
  • Enhancement: Do not render large artifacts and provide preview button.
  • Enhancement: Simplify switching to data view in metrics widgets.

Commercial

  • New: Add new notifications/alerts/events layer to the clients to allow sending mid-run events to external systems.
  • New: Add automations to monitor conditions and events and trigger actions.
  • New: Add control plane integrations.
  • New: Add an in-platform notification center to view and manage notifications and alerts.
  • New: Improve the scheduler to trigger runs based on schedules and events.
  • New: Add support for custom notifications and alerts.