N.B.: If you are upgrading from polyaxon v1.x, please check the v2 release notes and the migration guide.

V2.16.0

Core

  • New: Add plugins.tmux for reconnectable K8s exec sessions via tmux injection.
  • New: Add plugins.sandbox support for service operations.
  • New: Add the plx-exec sandbox runtime with process execution, background execution, PTY sessions, filesystem APIs, health checks, and activity tracking.
  • New: Add authenticated sandbox proxy routes for execution, filesystem, PTY, and SSH tunnel traffic.
  • New: Add SSH plugin support for service operations.
  • Fix: Issue in requirements compatibility issue with traceml version.
  • Fix: SDK configuration compatibility with kubernetes client v35 (missing tls_server_name).
  • Fix: Issue with tar path traversal protection in untar_file.
  • Fix: Clean orphan pipeline parents during scheduler reconciliation.
  • Fix: Preserve cached sandbox auth denials in streams.
  • Fix: Tune S3 filesystem upload defaults.
  • Enhancement: Expose tmux plugin state in run meta_info for CLI and UI consumption.
  • Enhancement: Auto-use tmux session in CLI shell and UI when the tmux plugin is enabled.
  • Enhancement: Add crons to clean up old stats.
  • Enhancement: Add a configurable global cleanup interval for runs stuck in warning states.
  • Enhancement: Add tiered stats snapshot retention with per-owner compaction tasks.
  • Enhancement: Resolve bookmarks with active content types.

CLI

  • New: Add async organization, project, and run clients.
  • New: Add async run logs, status polling, reference logging, artifact streaming, artifact downloads, and store downloads.
  • New: Add generated sandbox SDK surface and Python sandbox client helpers.
  • New: Add sandbox CLI commands for shell, exec, filesystem, and PTY workflows.
  • New: Add operation exec commands backed by the sandbox runtime.
  • New: Add SSH setup, config, tunnel, ProxyCommand, and connect passthrough commands.
  • Fix: Fix CLI shell inspection parsing to handle the pods wrapper in the inspect response.
  • Fix: Close owned agent clients on exit.
  • Fix: Fix SDK async configuration and lazy transport close behavior.
  • Fix: Align sandbox int64 fields with REST JSON.
  • Fix: Reconcile current Polyaxon CRDs in the Kubernetes executor.
  • Enhancement: Rewrite CLI PseudoTerminal to remove unused fork, fix cleanup, and improve error handling.
  • Enhancement: Auto-resolve namespaces in sandbox clients.
  • Enhancement: Pin SSH host keys through Polyaxon-managed known_hosts.

Operator

  • Fix: Stop failing services after the retry budget is exhausted.
  • Enhancement: Enrich job, service, and Kubeflow backoff failure messages.

Deployment

  • Enhancement: Improve post-installation instructions.
  • Enhancement: Extend init container with SSH and sandbox tools for service operations.
  • Enhancement: Remove stale agent replica configuration.
  • Fix: Fix serviceAccountImagePullSecrets rendering in Helm chart service accounts.

UI

  • Enhancement: Improve agent host validation logic.
  • Enhancement: Support space-separated commands in the shell component (in addition to comma-separated).
  • Enhancement: Auto-detect tmux plugin and set default shell command to tmux when enabled.
  • Enhancement: Wrap long connect modal commands.
  • Fix: Show SSH connection details or an empty state in the connect modal.
  • Fix: Stabilize smoothed line chart focus behavior.
  • Fix: Fix editing connections on Chrome.