V1DaskCluster

polyaxon._flow.run.dask.dask.V1DaskCluster()

Dask cluster specification for running distributed workloads. Dask cluster.

Creates a Dask cluster with scheduler and worker replicas. Platform automatically handles service, health checks, and dashboard configuration.

• Args:
  • kind: str, should be equal daskcluster
  • worker: V1DaskReplica, optional - Worker replica specification
  • scheduler: V1DaskReplica, optional - Scheduler replica specification
  • min_replicas: int, optional - Minimum number of workers for autoscaling
  • max_replicas: int, optional - Maximum number of workers for autoscaling

YAML usage

run:
  kind: daskcluster
  worker:
    replicas: 3
    container:
      image: daskdev/dask:latest
      resources:
        requests:
          memory: 2Gi
          cpu: 1
  scheduler:
    container:
      image: daskdev/dask:latest
      resources:
        requests:
          memory: 1Gi
          cpu: 1

Autoscaling

Enable autoscaling by setting minReplicas and maxReplicas. When both are set, a DaskAutoscaler resource will be created.

run:
  kind: daskcluster
  minReplicas: 1
  maxReplicas: 10
  worker:
    replicas: 2
    container:
      image: daskdev/dask:latest
  scheduler:
    container:
      image: daskdev/dask:latest