Polyaxon allows to schedule Pytorch experiments and Pytorch distributed experiments, and supports tracking metrics, outputs, and models natively.
Before you can use the pytorchjob runtime, you need to make sure that PytorchJob operator and the CRD (custom resource definition) are deployed in your cluster.
To be able to schedule distributed jobs with the PytorchJob operator, you need to enable the operator in your deployment config.
You need to enable the operator in Polyaxon CE deployment or Polyaxon Agent deployment:
operators: pytorchjob: true
Once you have the PytorchJob operator running on a Kubernetes namespace managed by Polyaxon, you can check the specification for creating components with the pytorchjob runtime:
version: 1.1 kind: component run: kind: pytorchjob ...
For more details about the specification for creating pytorchjob runtime, please check please check this section.
Running components with the pytorchjob runtime is similar to running any other component:
polyaxon run -f manifest.yaml -P ...
After running an operation with this component, you can view it on the Dashboard:
polyaxon ops dashboard
polyaxon ops dashboard -p [project-name] -uid [run-uuid] -y
To stop a running operation with this component:
polyaxon ops stop
polyaxon ops stop -p [project-name] -uid [run-uuid]
To run this component using Polyaxon Client:
from polyaxon.client import RunClient client = RunClient(...) client.create_from_polyaxonfile(polyaxonfile="path/to/file", ...)