Glossary

Specialized compute hardware used to speed up model training or inference, such as GPUs or TPUs.

Kubernetes scheduling rules that attract workloads to specific nodes or keep related workloads together based on labels.

A Polyaxon component that manages workload scheduling and execution on a Kubernetes cluster. Agents connect to the control plane to pull and run operations on their designated queues.

A visual representation of complex AI agent workflows in Polyaxon. Agent graphs help you understand and debug multi-step reasoning processes and agent interactions by displaying the flow of observations within a trace.

A queue served by a Polyaxon agent. It determines where submitted operations are picked up and executed.

A workflow or application behavior where a model can plan, call tools, observe results, and continue toward a goal.

A notification triggered when a condition is met, such as a failed operation, resource issue, or metric threshold.

A manual evaluation method that allows domain experts to review and add scores and comments to traces, observations, or sessions. Useful for building ground truth, systematic labeling, and team collaboration.

Credentials used to authenticate with the Polyaxon API and SDKs. API keys are associated with a user account or service account and managed in organization settings.

Any output file produced by a run — models, datasets, plots, logs, or other binary files. Artifacts are tracked, versioned, and stored in configured artifact stores.

The relationship between an artifact, the run that produced it, and any downstream models, datasets, or versions that depend on it.

A specific version of an artifact in the artifact registry. Versions track lineage, metadata, and the run that produced them.

The storage backend where Polyaxon writes run outputs, logs, models, datasets, and other generated files.

A transformer mechanism that lets a model weight relevant tokens and context when producing an output.

Immutable records of important organization, project, and system actions used for security, compliance, and incident investigation.

Running missed or historical scheduled operations for previous time windows, often after downtime or a workflow change.

A reference result used to compare new models, prompts, datasets, or optimization strategies.

Grouping multiple examples or requests together to improve hardware utilization and throughput.

A hyperparameter optimization strategy that uses a probabilistic model to intelligently select the next set of parameters to evaluate, converging faster than grid or random search.

A repeatable evaluation setup used to compare models, prompts, pipelines, or infrastructure choices.

A seat or unit of organization capacity used for Polyaxon Cloud pricing, including connected clusters, concurrent runs, queues, and active schedules.

A container image build step that runs before the main operation. Builds can use Kaniko or Docker to create custom images from Dockerfiles or buildpacks.

Stored intermediate data or metadata reused across operations to avoid repeating work.

An observation type that represents a link between different application steps, such as passing context from a retriever to an LLM call.

A prompt type that consists of an array of messages with specific roles (system, user, assistant). Useful for managing complete conversation structures and chat history.

A saved training state that can be used to resume training, evaluate intermediate progress, or register a model version.

The Polyaxon command-line interface. Used to submit runs, manage projects, check statuses, download artifacts, and interact with all platform features from the terminal.

A reusable, parameterized template that defines the runtime behavior of an operation — container image, command, inputs, outputs, and resource requirements.

A registry of reusable components that teams can share, version, and reference across projects. Components in the hub can be used by name in operations.

The number of operations or workflow steps allowed to run at the same time.

A Kubernetes object used to inject non-secret configuration into pods as environment variables or mounted files.

A configured external resource that runs can access — cloud storage (S3, GCS, Azure), git repos, Docker registries, or databases. Connections are defined per agent.

Runtime variables and metadata automatically available to an operation during execution. Includes run UUID, project info, parameter values, and connection details.

The maximum amount of input and generated text a model can consider in a single request.

The Polyaxon service layer that manages projects, users, metadata, scheduling decisions, and agent coordination.

Measuring and attributing compute, token, or infrastructure costs to runs, projects, users, or applications.

An evolutionary optimization operation that combines parts of two candidates to produce new candidates.

Flexible, self-service analytics dashboards that allow you to visualize and monitor metrics from your LLM application. Dashboards support multiple chart types, filtering, and multi-level aggregations.

A Directed Acyclic Graph pipeline that chains multiple operations with dependencies. Used to build multi-step ML workflows like data preparation, training, evaluation, and deployment.

Policies and actions for permanently removing selected metadata, logs, artifacts, or project data.

Rules that determine how long metadata, activity logs, archived operations, or generated data are kept.

A collection of test cases (dataset items) used to test and benchmark LLM applications. Datasets contain inputs and optionally expected outputs for systematic testing.

Also known as a Dataset Run. The execution of a dataset through your LLM application, producing outputs that can be evaluated. Links dataset items to their corresponding traces.

An individual test case within a dataset. Each item contains an input (the scenario to test) and optionally an expected output.

A specific version of a dataset used to make experiments and evaluations reproducible.

Running training across multiple workers or GPUs using frameworks like Horovod, PyTorch DDP, TensorFlow, or MPI. Polyaxon handles worker coordination and resource allocation.

An observation type that represents a call to an LLM to generate embeddings. Can include model information, token usage, and costs.

A way to organize traces, observations, and scores from different deployment contexts (e.g., production, staging, development). Helps keep data separate while using the same project.

Temporary local storage attached to a running pod and removed when the pod is deleted.

The process of measuring model, prompt, or application quality using metrics, datasets, human review, or automated scoring.

A function that scores traces, observations, sessions, or dataset runs. Methods include LLM-as-a-Judge for subjective assessments, Annotation Queues for human review, Scores via UI for spot checks, and Scores via API/SDK for programmatic evaluation.

An observation type that represents functions assessing the relevance, correctness, or helpfulness of LLM outputs. Also refers to the function that scores experiment results.

A trigger condition in Polyaxon's automation system. Events fire when specific conditions are met (e.g., run completes, metric threshold crossed) and can trigger downstream operations or hooks.

A prompt optimizer that explores many candidate prompt structures using mutation, crossover, and selection.

A prompt optimizer that searches over few-shot example count, order, or selection using Bayesian optimization.

Training a pretrained model further on a specific dataset to adapt it to a task, style, or domain.

The component of Polyaxon that manages DAG execution — resolving dependencies, passing outputs between steps, handling failures, and parallelizing independent operations.

The process of sending buffered trace data to the Polyaxon server. Important for short-lived applications to ensure no data is lost when the process terminates.

A model capability for returning structured arguments that an application can use to call external functions or tools.

An observation type that logs outputs from AI models including prompts, completions, token usage, and costs. The most common observation type for LLM calls.

A reflection-based prompt optimization approach that searches for Pareto-improving prompts across minibatches.

A graphics processing unit used as an accelerator for model training, inference, and data processing workloads.

A hyperparameter optimization strategy that exhaustively evaluates every combination of specified parameter values. Simple but can be computationally expensive.

An observation type that represents a component protecting against malicious content, jailbreaks, or other security risks.

High-bandwidth memory used by GPUs and accelerators for fast access to model weights, activations, and tensor data.

A probe or signal used to determine whether a service, agent, or workload is ready and functioning.

An action triggered by an event during a run's lifecycle — such as sending a notification, starting a downstream operation, or calling a webhook when a run succeeds or fails.

A prompt optimization method focused on batching failures, synthesizing root-cause themes, and proposing targeted fixes.

Evaluation performed by people reviewing outputs, traces, or experiment results against criteria.

An adaptive hyperparameter optimization strategy that uses early stopping to quickly discard underperforming configurations, focusing compute on the most promising candidates.

A configuration value that controls the training process (e.g., learning rate, batch size, number of layers). Polyaxon supports automated tuning via grid, random, Bayesian, and Hyperband strategies.

Automated search over training or runtime parameters to maximize an objective metric.

The practice of adjusting hyperparameters such as learning rate, batch size, or model depth to improve results.

A Kubernetes secret used by pods to authenticate with private container registries.

Running a trained model to generate predictions, completions, or responses from new inputs.

A high-throughput, low-latency network interconnect often used for distributed GPU training and inference clusters.

The process of adding code to record application behavior. Polyaxon provides context managers, observe wrappers, and manual observation methods for instrumenting your application.

The network fabric connecting accelerators or nodes for distributed training, inference, and data movement.

A run type that executes a container to completion and then terminates. The most common run kind for training scripts, data processing, and batch operations.

A mechanism to aggregate outputs from multiple upstream runs and pass them as inputs to a downstream operation. Used in DAGs to combine results from parallel branches.

The container orchestration platform used by Polyaxon to schedule and run workloads.

Cached key and value tensors from attention layers that are reused during generation to avoid recomputing prior context.

The time taken for a request, operation, model call, or pipeline step to complete.

A large language model that generates or completes text and can power prompts, agents, evaluation, and AI applications.

An API key configuration that allows Polyaxon to call LLM models in the Playground or for LLM-as-a-Judge evaluations. Supports providers like OpenAI, Anthropic, and Google.

An evaluation method that uses an LLM to score the output of your application based on custom criteria. Provides scalable, repeatable evaluations with chain-of-thought reasoning.

Shows all observations concatenated. Great for quickly scanning through them.

Text output emitted by applications, containers, services, or platform components during execution.

A strategy that fans out a single operation into multiple parallel runs, each receiving a different set of parameters from a predefined list. Similar to a map operation in functional programming.

A strategy for generating multiple operations from combinations of parameters, often used for sweeps or tuning.

A Model Context Protocol server that enables AI-powered tools to interact with Polyaxon data. Used for advanced integrations and AI-assisted workflows.

A prompt optimizer that uses a reasoning model to critique and rewrite prompts against a dataset and metric.

A numerical value logged during a run to track training progress — loss, accuracy, F1 score, etc. Metrics can be logged at each step and visualized as time-series charts.

An API endpoint for retrieving customized analytics from Polyaxon data. Allows specifying dimensions, metrics, filters, and time granularity to build custom reports and dashboards for LLM applications.

A configuration that stores pricing information for an LLM model. Model definitions specify the cost per input and output token, enabling Polyaxon to automatically calculate the price of generations based on token usage.

Splitting a model across multiple devices or nodes to fit larger models or speed up training and inference.

A centralized catalog for versioning, staging, and promoting trained models. Models move through stages (staging, production, archived) with full lineage back to the training run.

A lifecycle label for a model version, such as staging, production, archived, or custom stages.

A specific version of a model in the model registry. Each version tracks the run that produced it, metrics at time of registration, and a promotion stage.

An evolutionary optimization operation that changes part of a candidate to explore nearby alternatives.

A Kubernetes isolation boundary for grouping resources such as pods, secrets, services, and quotas.

A Kubernetes worker machine that provides CPU, memory, storage, and accelerators for running pods.

A Kubernetes scheduling constraint that places a pod on nodes matching specific labels.

A message sent to users or external systems when an event or alert occurs.

The metric or goal an optimization process tries to maximize or minimize.

An individual step within a trace. Observations can be of different types (span, generation, event, tool, etc.) and can be nested to represent hierarchical workflows.

A basic observation type used to track discrete events in a trace. Events are the building blocks of tracing.

Testing your application against a fixed dataset before deployment. Used to validate changes and catch regressions during development.

Scoring live production traces to catch issues in real traffic. Helps identify edge cases and monitor application quality in production.

An open standard for collecting telemetry data from applications. Polyaxon is built on OpenTelemetry, enabling interoperability and reducing vendor lock-in.

A concrete instance of a component with bound parameters, queue assignment, and scheduling configuration. Operations are what you submit to Polyaxon to create runs.

The subsystem that manages automated hyperparameter searches. Supports grid, random, Bayesian optimization, and Hyperband strategies with configurable objectives and early stopping.

A top-level entity in Polyaxon that contains projects, teams, and agents. Organizations manage billing, team members, SSO, and resource quotas.

An optimizer that leaves the prompt text unchanged and searches over sampling parameters such as temperature or top-p.

A multi-objective search strategy that keeps candidates that improve one objective without making another objective worse.

A Kubernetes storage resource that can outlive individual pods and be mounted by workloads.

A sequence or graph of operations that transforms data, trains models, evaluates outputs, or deploys artifacts.

A model-parallel training or inference strategy where layers are split across devices and microbatches move through the pipeline.

The LLM Playground where you can test, iterate, and compare different prompts and models directly in Polyaxon without writing code.

The smallest Kubernetes execution unit, containing one or more containers that share networking and storage.

A YAML specification file that declares components, operations, and workflows. The primary interface for defining what Polyaxon should run, including container, resources, inputs, outputs, and scheduling.

Evicting lower-priority workloads so higher-priority workloads can run when cluster resources are constrained.

A reusable set of scheduling defaults (queue, resource limits, node selectors, tolerations) that can be applied to any operation. Avoids repeating common infrastructure config.

A scheduling signal used to decide which operations should run first when resources are limited.

A Kubernetes object that assigns scheduling priority to pods and can influence preemption.

A container that groups runs, artifacts, models, and components within an organization. Projects enable fine-grained access control and organize work by team or application.

Moving a model, artifact, or component version to a new lifecycle stage such as staging or production.

A label that can be assigned to a prompt version. Used to mark prompt versions as production or staging to fetch them via the SDK or API.

A systematic approach to storing, versioning, and retrieving prompts for LLM applications. Decouples prompt updates from code deployment.

A metric used to evaluate prompt behavior, such as accuracy, helpfulness, safety, cost, latency, or response length.

Automated or guided improvement of prompts using datasets, metrics, model feedback, or search algorithms.

Placeholders in prompts that are dynamically filled at runtime. Allow creating reusable prompt templates with customizable content.

Restricts the ability to modify certain prompt labels (e.g. production) from being added to new prompt versions to admins and owners. This prevents accidental or unauthorized changes to production prompts.

The REST API that provides programmatic access to all Polyaxon features — runs, artifacts, models, projects, and metrics. Used for custom integrations and automation.

A PersistentVolumeClaim, which requests durable Kubernetes storage for a workload.

Polyaxon's filtering syntax for searching and filtering runs, metrics, and artifacts. Supports comparisons, logical operators, and field-specific queries.

A scheduling queue managed by an agent. Queues control concurrency limits, priorities, and resource allocation for operations submitted to them.

A limit on resource usage for a namespace, organization, project, queue, or team.

Retrieval-Augmented Generation, a pattern that retrieves external context before generating a model response.

A hyperparameter optimization strategy that samples parameter combinations randomly from defined distributions. Often more efficient than grid search for high-dimensional spaces.

Role-Based Access Control that manages permissions within Polyaxon. Roles include Owner, Admin, Member, Viewer, and None, each with specific scopes at the organization and project level.

Testing an AI system with adversarial or high-risk inputs to uncover safety, security, or reliability failures.

A small batch of examples used by reflection-based optimizers to critique behavior and propose prompt improvements.

A system for organizing, versioning, and promoting models, artifacts, components, or datasets.

A webhook-based trigger that allows running SDK experiments from the Polyaxon UI. Configure a webhook URL and default config, then trigger experiments that fetch the dataset, run your application, and ingest scores back into Polyaxon.

The ability to recreate a result from recorded code, parameters, environment, data, artifacts, and lineage.

Maximum compute resources a workload is allowed to consume, such as CPU, memory, GPU, or ephemeral storage.

Compute resources reserved for a workload so the scheduler can place it on a suitable node.

Measured consumption of compute, memory, storage, network, or accelerator resources over time.

An observation type that represents data retrieval steps, such as calls to vector stores or databases in RAG applications.

A single execution of an operation. Each run has a unique UUID, tracked inputs (parameters), outputs (metrics, artifacts), status, and logs. Runs can be jobs, services, DAGs, or matrix operations.

Generation settings that influence model output, such as temperature, top-p, max tokens, or stop sequences.

A cron, interval, or datetime trigger that automatically submits operations at specified times. Used for recurring training jobs, data pipelines, or periodic evaluations.

System for Cross-domain Identity Management, a standard for provisioning and deprovisioning users and groups.

The output of an evaluation. Scores can be numeric, categorical, or boolean and are assigned to traces, observations, sessions, or dataset runs.

A configuration defining how a score is calculated and interpreted. Includes data type, value constraints, and categories for standardized scoring.

Software Development Kit. Polyaxon provides a Python SDK for tracking experiments, managing runs, and accessing the API programmatically.

The set of candidate values, prompts, examples, or parameters an optimizer is allowed to explore.

A Kubernetes object for storing sensitive values such as tokens, passwords, certificates, or registry credentials.

A run type that starts a long-running process with a network endpoint — Jupyter notebooks, TensorBoard, Streamlit apps, REST APIs, or custom dashboards.

An identity used by applications, agents, or automation to authenticate without using a personal user account.

A way to group related traces that are part of the same user interaction. Commonly used for multi-turn conversations or chat threads.

An observation type that represents the duration of a unit of work in a trace. The default observation type for most operations.

Discounted cloud compute capacity that can be interrupted by the provider, often used for fault-tolerant workloads.

Single Sign-On, an authentication setup that lets users access Polyaxon through an external identity provider.

Flexible labels that categorize and filter runs. Useful for organizing experiments by purpose, dataset, model architecture, or team.

A function definition that processes dataset items during an experiment. The task represents the application code you want to test.

A group of users used to manage access, roles, ownership, and collaboration within an organization.

A sampling parameter that controls randomness in model outputs; higher values generally produce more varied responses.

A model-parallel strategy where tensor operations or layer shards are split across multiple devices.

A single input or scenario used to evaluate a model, prompt, pipeline, or application.

A prompt type that consists of a single string. Ideal for simple use cases or when you only need a system message.

The amount of work processed per unit time, such as requests per second, tokens per second, or completed operations.

The basic unit of text that LLMs process. Tokens can be words, parts of words, or characters depending on the model's tokenizer. Token counts determine API costs and context window limits. Polyaxon tracks input and output tokens for cost monitoring and optimization.

The number of input and output tokens consumed by model calls, often used for cost and performance analysis.

Kubernetes scheduling settings that allow pods to run on nodes with matching taints.

An observation type that represents a tool call in your application, such as calling a weather API or executing a database query.

A model or application step that invokes an external tool, function, API, database, or service.

A sampling parameter that limits generation to the smallest set of tokens whose cumulative probability reaches a threshold.

A tensor processing unit, an accelerator designed for machine learning workloads.

A single request or operation in your LLM application. Traces contain the overall input, output, and metadata, along with nested observations that capture each step.

The process of capturing structured logs of every request in your LLM application. Includes prompts, responses, token usage, latency, and any intermediate steps.

The process of logging metrics, parameters, artifacts, and metadata during experiment runs. Polyaxon's tracking API captures everything needed to reproduce and compare experiments.

One evaluated candidate in an optimization process, such as a parameter configuration, prompt version, or example set.

The ability to associate traces with users via a userId. Enables per-user analytics, cost tracking, and filtering.

A database or index that stores embeddings and supports similarity search for retrieval workflows.

Charts, plots, and dashboards for comparing runs and analyzing metrics. Includes built-in support for TensorBoard, Plotly, and custom visualization plugins.

A Kubernetes batch scheduling system often used for queueing, gang scheduling, and ML workloads.

A queue managed by Volcano for scheduling batch and ML workloads on Kubernetes.

GPU memory used to store model weights, activations, batches, and KV cache during training or inference.

An HTTP callback sent to an external system when an event occurs.

A coordinated set of operations that run in sequence, in parallel, or as a dependency graph.