Note: This is an advanced use-case that you might want to skip for now and come back to later.

We previously learned how to configure the CLI to iterate with:

• local code
• git repo

In both guides, we had to adjust the polyaxonfile to fit a specific workflow. Often, users will probably need to use both workflows to iterate on their experiments, for quick changes they will need to upload code directly before starting a run, and after committing the changes they will need to trigger based on the git repo, finally, they might need to have the manifest compatible with a fork of their main repo or a repo that is named differently.

In this guide, we will learn how to improve our Polyaxonfile to adapt it to different situations with presets.

Run the commands from the root of the polyaxon-quick-start repository used by the previous local-code and Git guides. If you are starting here directly, complete the local-code setup first. The Git example also uses the polyaxongit.yaml created in the Git guide.

Remove the git initializer from the component

In order to make the component work in different situations, i.e. get executed based on an initialized git repo or from local code, we will start by removing the git initializer and the container.workingDir section:

version: 1.1
kind: component
name: typed-experiment
description: experiment with inputs
tags: [examples]

inputs:
- {name: conv1_size, type: int, value: 32, isOptional: true}
- {name: conv2_size, type: int, value: 64, isOptional: true}
- {name: dropout, type: float, value: 0.2, isOptional: true}
- {name: hidden1_size, type: int, value: 500, isOptional: true}
- {name: conv_activation, type: str, value: relu, isOptional: true}
- {name: dense_activation, type: str, value: relu, isOptional: true}
- {name: optimizer, type: str, value: adam, isOptional: true}
- {name: learning_rate, type: float, value: 0.01, isOptional: true}
- {name: epochs, type: int}
outputs:
- {name: loss, type: float}
- {name: accuracy, type: float}

run:
  kind: job
  container:
    image: polyaxon/polyaxon-quick-start
    command: ["python3", "model.py"]
    args: [
      "--conv1_size={{ conv1_size }}",
      "--conv2_size={{ conv2_size }}",
      "--dropout={{ dropout }}",
      "--hidden1_size={{ hidden1_size }}",
      "--optimizer={{ optimizer }}",
      "--conv_activation={{ conv_activation }}",
      "--dense_activation={{ dense_activation }}",
      "--learning_rate={{ learning_rate }}",
      "--epochs={{ epochs }}"
    ]

You can view this file under scheduling/presets/. The same folder contains the other files used with this component.

If we try to execute this component, it will fail, since the container cannot resolve the file model.py.

We can additionally make this manifest fail fast at the CLI level, in other terms, we can prevent users from attempting to submit this file, wait for the operation to be scheduled, and then check the logs to find out that the container is missing the code. Polyaxon's specification has a section called template that we can add to any polyaxonfile to prevent running it without proper initialization:

template:
  enabled: true
  description: "This polyaxonfile requires a preset to run correctly."
  fields: ["runPatch.container.workingDir"]

Note: You can learn more about the template section here

Using the local code preset

In this subsection we will run scheduling/presets/polyaxonfile.yaml with scheduling/presets/upload-workingdir-preset.yaml:

polyaxon run \
  -f scheduling/presets/polyaxonfile.yaml \
  -f scheduling/presets/upload-workingdir-preset.yaml \
  -u-from scheduling \
  -l

By running this command we will supplement the main component with the additional section that resolves the path to model.py from the uploaded code:

runPatch:
  container:
    workingDir: "{{ globals.run_artifacts_path }}/uploads"

Using the git integration

In this subsection we will run scheduling/presets/polyaxonfile.yaml with scheduling/presets/git-workingdir-preset.yaml and the --git-preset flag:

In this case we use the Git preset configured in the previous guide and supplement the main component with the section that resolves the path to model.py from the initialized code:

polyaxon run \
  -f scheduling/presets/polyaxonfile.yaml \
  -f scheduling/presets/git-workingdir-preset.yaml \
  --git-preset \
  -l

By running this command we will supplement the main component with the additional section that resolves the path to model.py from the git integration:

runPatch:
  container:
    workingDir: "{{ globals.artifacts_path }}/polyaxon-quick-start/scheduling"

Using the custom git preset and git workingDir

In this subsection we will run scheduling/presets/polyaxonfile.yaml with scheduling/presets/git-init.yaml and scheduling/presets/git-workingdir-preset.yaml:

This is similar to the previous subsection. The difference is that we do not use --git-preset; instead, we provide scheduling/presets/git-init.yaml:

polyaxon run \
  -f scheduling/presets/polyaxonfile.yaml \
  -f scheduling/presets/git-workingdir-preset.yaml \
  -f scheduling/presets/git-init.yaml \
  -l

By running this command we will supplement the main component with the additional section that resolves the path to model.py from the git preset:

runPatch:
  container:
    workingDir: "{{ globals.artifacts_path }}/polyaxon-quick-start/scheduling"

More presets

In this guide we only extracted the logic for initializing/uploading the code and for setting the workingDir, but presets are a very powerful feature in Polyaxon, it allows users to define several reusable configurations, e.g. complex node selectors, resources requirements, ...

You can check the helpers folder for additional preset examples.

For more details about using presets, see the scheduling presets guide.