Polyaxon provides several ways to optimize your pipelines, speeds up your hyperparameter tuning, and early terminate workflows and underperforming runs.

V1FailureEarlyStopping

polyaxon._flow.early_stopping.policies.V1FailureEarlyStopping()

Failure early stopping is an early stopping strategy based on statuses of runs that allows to terminate a dag, a mapping, or hyperparameter tuning group when they reach a certain level of failures.

If a percentage of the runs in the pipeline fail, the pipeline will be marked as failed as well, and all pending or running operations will be stopped.

  • Args:
    • kind: str, should be equal to failure_early_stopping
    • percent: int (>0, <=99)

YAML usage

earlyStopping:
  - kind: failure_early_stopping
    percent: 50

Python usage

from polyaxon.schemas import V1FailureEarlyStopping
early_stopping = [V1FailureEarlyStopping(percent=50)]

Fields

kind

The kind signals to the CLI, client, and other tools that this early stopping is failure_early_stopping.

If you are using the python client to create the early stopping, this field is not required and is set by default.

earlyStopping:
  - kind: failure_early_stopping

percent

The percentage of failed runs at each evaluation interval, should be a value between 1 and 99.

earlyStopping:
  - kind: failure_early_stopping
    percent: 30

V1MetricEarlyStopping

polyaxon._flow.early_stopping.policies.V1MetricEarlyStopping()

Metric early stopping is an early stopping strategy based on metrics of runs, it allows to terminate a dag, a mapping, or hyperparameter tuning when a run’s metric(s) meet(s) one or multiple conditions.

If no policy is set and a metric early stopping condition is met the pipeline will be marked as succeeded and all pending or running operations will be stopped.

If a policy is set only the runs that validate the policy will be stopped.

  • Args:
    • kind: str, should be equal to metric_early_stopping
    • metric: str
    • value: float
    • optimization: Union[“maximize”, “minimize”]
    • policy: Union[V1MedianStoppingPolicy, V1TruncationStoppingPolicy, V1DiffStoppingPolicy], optional

YAML usage

earlyStopping:
  - kind: metric_early_stopping
    metric: loss
    value: 0.001
    optimization: minimize
  - kind: metric_early_stopping
    metric: accuaracy
    value: 0.9
    optimization: maximize

Python usage

from polyaxon.schemas import V1MetricEarlyStopping, V1Optimization
early_stopping = [
    V1MetricEarlyStopping(metric="loss", optimization=V1Optimization.MINIMIZE),
    V1MetricEarlyStopping(metric="accuracy", optimization=V1Optimization.MAXIMIZE),
]

Fields

kind

The kind signals to the CLI, client, and other tools that this early stopping is metric_early_stopping.

If you are using the python client to create the early stopping, this field is not required and is set by default.

earlyStopping:
  - kind: metric_early_stopping

metric

The metric to track for checking the early stopping condition. This metric should be logged using one of the tracking modules or API.

earlyStopping:
  - metric: loss

value

The metric value for checking the early stopping condition.

earlyStopping:
  - value: 0.5

optimization

The optimization defines the goal or how to measure the performance of the defined metric.

earlyStopping:
  - optimization: maximize

policy

A policy allows to defines how to evaluate the metric value against the defined value, there are a couple of policies:

  • MedianStopping: Early stopping with median stopping, this policy computes running medians across all runs and stops those whose best performance is worse than the median of the running runs.
  • DiffStopping: Early stopping with diff factor stopping, this policy computes checked runs against the best run and stops those whose performance is worse than the best by the factor defined by the user.
  • TruncationStopping: Early stopping with truncation stopping, this policy stops a percentage of all running runs at every evaluation.