See it in code
Train. Evaluate. Approve the handoff.
Connect your training and evaluation components, check the recorded score, and require approval before registering acme/model-reg:v1.
Configure dependencies and conditionsversion: 1.1
kind: component
name: reviewed-model
run:
kind: dag
operations:
- name: train
hubRef: acme/train-model:v1
- name: evaluate
hubRef: acme/evaluate-model:v1
params:
run: {ref: ops.train, value: globals.uuid}
- name: register
hubRef: acme/register-model:v1
isApproved: false
trigger: all_succeeded
conditions: "{{ accuracy >= 0.9 }}"
params:
run: {ref: ops.train, value: globals.uuid}
accuracy: {ref: ops.evaluate, value: outputs.val_accuracy, contextOnly: true}
registry: {value: acme/model-reg}
version: {value: v1}Replace the illustrative acme projects and components with your own references. The evaluator declares and records val_accuracy; the training run records model and env artifacts. The registration component accepts run, registry, and version inputs. Replace the run UUID placeholders in the approval commands.
Reuse operations instead of coordinating scripts by hand
A Polyaxon DAG connects operations through dependencies and input/output references. A preparation job can produce the inputs for training; an evaluation job can consume the resulting model. Each operation keeps its own configuration, status, logs, and outputs.
The steps can use different runtimes, including jobs, distributed jobs, services, or nested workflows. The DAG coordinates them; the components and their containers define the actual work.
In the example above, evaluate receives the training run's UUID. The register operation receives that same UUID, uses the evaluator's val_accuracy output as a condition, and requires approval to create acme/model-reg:v1.
Make continuation a deliberate choice
Use a trigger to decide which upstream statuses permit the next operation. Add a condition when the decision depends on a recorded output, such as a validation score. A job succeeding is not the same as a model meeting your quality requirement.
For a human review, mark the relevant operation as requiring approval. Its dependent branch waits until that operation is approved. Define the promotion or publication step explicitly; Polyaxon does not decide what makes a model acceptable.
Once the candidate is reviewed, the registration component selects the training run's model and env artifacts. Continue with the Registry example to see the registration and consumption calls.
Schedule recurring work and bound parallel runs
Run an operation or DAG on a cron or interval schedule. Where runs should not overlap, configure the schedule to depend on its previous execution. A parameter sweep can explore multiple configurations while a workflow concurrency limit bounds its parallel work.
The documented automation example starts an experiment, runs a tuning process when loss exceeds a configured threshold, and collects the best results. It shows how conditions, sweeps, and output references fit together without requiring a separate script to submit every run.
Explore hyperparameter tuning · Walk through a conditional DAG · Configure a schedule · Limit concurrency
Inspect failures before choosing how to recover
Follow the failed operation's logs, outputs, and lineage to determine whether the input, code, or environment needs to change. Resume an operation when it should reuse its prior artifacts, or restart it when you need a separate run record. Copy mode can load artifacts into a restarted run.
Your application must implement checkpoint loading for training to resume. Review the downstream dependencies before retrying work that writes to external systems: restarting an operation does not roll back those writes or make them safe to repeat.