Polyaxon mapping is an orchestration tool to apply a list of parameter combinations to a component sequentially or in parallel. When a mapping is defined, Polyaxon creates a pipeline to watch the generated executions and manage their lifecycle and concurrency.

Overview

One major difference between mapping and other hyperparameter search options, is that the user is responsible for providing the list of configurations. The mapping can be used to receive a list of configurations based on an external service that generates suggestions.

Because mapping generates a pipeline, the pipeline helpers can be used for managing concurrency, early stopping, caching, and similar behavior.

Use cases

Polyaxon mapping can be used to:

• Run a component in parallel using a user-defined list of parameters
• Train a machine learning model using an external optimization and suggestion service
• Leverage a map/reduce style pattern

Mapping

Sometimes you may want to run parallel executions and provide your own suggestions instead of using an algorithm provided by Polyaxon. Mapping is how you can provide a predefined space or a list of inputs.

Mapping can be used as well to parallelize a job for fetching data or loading information from a source to a destination concurrently.

The mapping.yaml polyaxonfile defines all the values we want to try:

version: 1.1
kind: operation
matrix:
  kind: mapping
  values:
  - learning_rate: 0.001
    dropout: 0.25
    conv_activation: relu
    epochs: 5
  - learning_rate: 0.01
    dropout: 0.5
    conv_activation: sigmoid
    epochs: 5
urlRef: https://raw.githubusercontent.com/polyaxon/polyaxon-quick-start/master/experimentation/typed.yaml

Running an operation with a mapping is also similar to any other operation:

polyaxon run --url https://raw.githubusercontent.com/polyaxon/polyaxon-quick-start/master/optimization/mapping.yaml

If you have cloned the quick-start repo, you can run:

polyaxon run -f optimization/mapping.yaml

For more details check the mapping specification.

Learn More

You can also learn about tools to control the cache of executions with similar configurations, and concurrency for enforcing of parallelism. Finally, every pipeline in Polyaxon can also define early stopping strategies.