Events

Events are one the last major addition to the Polyaxonfile specification in this release, users can now run DAGs with full events support.

Most workflow orchestrators support an aggregated upstream condition, e.g. all succeeded, all failed, or all done, basically an orchestrator would schedule an operation or a task only after the upstream is finished. That was also the case for Polyaxon until this release.

Events allow starting an operation in response to any event generated by an upstream entity. In this release, the entity is an operation running in the same DAG context. In the future, the entity could be anything from a Git commit, a new S3 blob, to an internal alert, or a new registered model version.

Here’s an example of starting a Tensorboard as soon as a training operation starts running:

version: 1.1
kind: component
name: experiment-with-tensorboard
run:
  kind: dag
  operations:
    - name: experiment
      pathRef: './experiment.yml'
      params:
        learning_rate:
          value: 0.005
        epochs:
          value: 10
    - name: tensorboard
      hubRef: tensorboard
      termination:
        timeout: 7200
      params:
        uuid:
          ref: ops.experiment
          value: globals.uuid
      events:
        - ref: ops.experiment
          kinds: [run_status_running]

This DAG will schedule two operations, a job for training a DL experiment and a Tensorboard to visualize the outputs of the experiment. Instead of waiting for the experiment to finish before starting a Tensorboard, or copying the UUID of the job to start the Tensorboard manually, this DAG will schedule the Tensorboard automatically as soon as the training starts running. If the training fails because of some compilation error, the Tensorboard will be marked as skipped.

Learn More about Polyaxon

This blog post just goes over a couple of features that we shipped since our last product update, there are several other features and fixes that are worth checking. To learn more about all the features, fixes, and enhancements, please visit the release notes.

Polyaxon continues to grow quickly and keeps improving and providing the simplest machine learning abstraction. We hope that these updates will improve your workflows and increase your productivity, and again, thank you for your continued feedback and support.