Now that our component works, we want to be able to run it continuously. Polyaxon provides an abstraction to put operations on a schedule.
Running components on schedule
Schedules can be used at the operation level, and similar to the matrix section, they create a pipeline to follow the scheduling of a component or an operation.
The file schedules/recurrent_training.yaml
contains an operation definition with a schedule attached:
version: 1.1
kind: operation
name: every monday check
schedule:
kind: cron
cron: "0 0 * * MON"
params:
schedule_at:
value: '{{ globals.schedule_at }}'
contextOnly: true
optimizer: {value: sgd}
epochs: {value: 1}
urlRef: https://raw.githubusercontent.com/polyaxon/polyaxon-quick-start/master/experimentation/typed.yaml
This schedule is of kind cron
, and it will start an experiment every monday.
If you need to start this schedule at a specific date you can set the startAt
field, and to provide a stopping condition you can provide endAt
or maxRuns
.
N.B.: We are providing the the
schedule_at
variable to ensure that the operations are not marked ascache hit
and without disabling the cache.
It’s also possible to put the complete DAG that we created in the previous guide on schedule, similarly to this operation,
you can add a valid schedule section, this way, not only you automate the journey of creating, training, and validating a model, but also you do it continuously,
the file schedules/recurrent_dag.yaml
contains an example of such operation:
version: 1.1
kind: operation
name: every-monday-check
schedule:
kind: cron
cron: "0 0 * * MON"
params:
optimizer: { value: sgd }
epochs: { value: 1 }
cache:
disable: true
pathRef: ../dags/dag.yaml
N.B.: In this second example we are disabling the cache completely.
Learn More
You can check the schedules section for more details about all the possible schedules and their specifications.