ops-pull

Pulling runs

With the new v1.17 release, users can archive their runs to a local path by using:

polyaxon ops pull --help

Usage: polyaxon ops pull [OPTIONS]

  Pull a remote run or multiple remote runs to a local path.

  Uses /docs/core/cli/#caching

  Examples:

  $ polyaxon ops pull -uid 8aac02e3a62a4f0aaa257c59da5eab80

  $ polyaxon ops pull -uid 8aac02e3a62a4f0aaa257c59da5eab80 --no-artifacts

  $ polyaxon ops pull -uid 8aac02e3a62a4f0aaa257c59da5eab80 --path /tmp/base

  $ polyaxon ops pull -q "status: succeeded, kind: job, metrics.loss: <0.2" --l 10 --path /tmp/base

  $ polyaxon ops pull -a

Options:
  -p, --project TEXT      The project name, e.g. 'mnist' or 'acme/mnist'.
  -uid, --uid TEXT        The run uuid.
  -a, --all-runs          To pull all runs.
  -q, --query TEXT        To filter the runs based on a query spec.
  -l, --limit INTEGER     To limit the list of runs.
  -off, --offset INTEGER  To offset the list of runs.
  --no-artifacts          To disable downloading the run's artifacts and only
                          persist the metadata. This is useful if you want to
                          move a run from one Polyaxon deployment to another
                          while keeping the same artifacts store.
  --path PATH             Optional path where the runs are persisted, default
                          value is taken from the env var:
                          `POLYAXON_OFFLINE_ROOT`.
  --help                  Show this message and exit.

With this new command users can achieve the following use-cases:

  • Archiving and sharing a completed run (metadata, lineage information, artifacts, and outputs)
polyaxon ops pull -uid 8aac02e3a62a4f0aaa257c59da5eab80 -p owner/project
  • Archiving and sharing all completed runs (metadata, lineage information, artifacts, and outputs)
polyaxon ops pull -p owner/project --all
  • Archiving and sharing some completed runs (metadata, lineage information, artifacts, and outputs)
polyaxon ops pull -p owner/project --query "status: succeeded, metrics.loss: <0.1" --sort "metrics.loss" --limit 10
  • Archiving the run’s metadata and lineage information only
polyaxon ops pull -uid 8aac02e3a62a4f0aaa257c59da5eab80 -p owner/project --no-artifacts

the last command is useful when the user intends to transfer the metadata of one run or multiple runs from one cluster to another while using the same artifacts store.

Pushing local runs

Similar to pulling runs, users can also push local runs to a remove Polyaxon deployment by using:

polyaxon ops push --help

Usage: polyaxon ops push [OPTIONS]

  Push an local run to a remove server.

  Uses /docs/core/cli/#caching

  Examples:

  $ polyaxon ops push -a --clean

  $ polyaxon ops push --path /tmp/experiments --clean

  $ polyaxon ops push -uid 8aac02e3a62a4f0aaa257c59da5eab80 --no-artifacts

  $ polyaxon ops push -uid 8aac02e3a62a4f0aaa257c59da5eab80 --clean

  $ polyaxon ops push -uid 8aac02e3a62a4f0aaa257c59da5eab80 --reset-project

  $ polyaxon ops push -uid 8aac02e3a62a4f0aaa257c59da5eab80 --reset-project -p send-to-project

Options:
  -p, --project TEXT  The project name, e.g. 'mnist' or 'acme/mnist'.
  -uid, --uid TEXT    The run uuid.
  -a, --all-runs      To push all runs.
  --no-artifacts      To disable uploading artifacts and only sync metadata.
                      This is useful if you want to move a run from one
                      Polyaxon deployment to another while keeping the same
                      artifacts store (no artifacts transfer).
  -c, --clean         To clean the run(s) local data after syncing.
  --path PATH         Optional path where the runs are persisted, default
                      value is taken from the env var:
                      `POLYAXON_OFFLINE_ROOT`.
  --reset-project     Optional, to ignore the owner/project of the local run
                      and use the owner/project provided or resolved from the
                      current project.
  --help              Show this message and exit.
  • Push all runs under the local path and clean the path after pushing the run:
polyaxon ops push -a -c
  • Push a specific run under the local path
polyaxon ops push -uid 8aac02e3a62a4f0aaa257c59da5eab80 -c
  • Push a specific run under the local path without pushing the related artifacts, using if the artifacts store where the original data is located is accessible from the remove deployment:
polyaxon ops push -uid 8aac02e3a62a4f0aaa257c59da5eab80 --no-artifacts -c

Learn More about Polyaxon

This blog post just goes over a couple of features that we shipped in v1.17. To learn more about all the features, fixes, and enhancements, please visit the release notes and the short term roadmap.

Polyaxon continues to grow quickly and keeps improving and providing the simplest machine learning abstraction. We hope that these updates will improve your workflows and increase your productivity, and again, thank you for your continued feedback and support.