Data StoresData on Node Host Path
Polyaxon & Data on Node Host Path
How to use Polyaxon and Data on Node Host Path together
+
You can use one or multiple node host paths (host node's filesystem) to access data directly on your machine learning experiments and jobs
Tip: Please visit the Kubernetes documentation to learn about host paths.
Use the host path as a connection
connections:
- name: node-dataset
kind: host_path
schema:
mountPath: "/path/dataset"
hostPath: "/path/to/dataset"To mount the data with the read-only option:
connections:
- name: node-dataset
kind: host_path
schema:
mountPath: "/path/dataset"
hostPath: "/path/to/dataset"
readOnly: trueIf you want to access multiple datasets:
connections:
- name: dataset1
kind: host_path
schema:
mountPath: /plx-dataset1
hostPath: "/path/to/dataset1"
readOnly: true
- name: dataset2
kind: host_path
schema:
mountPath: /plx-dataset2
hostPath: "/path/to/dataset2"
readOnly: trueUpdate/Install Polyaxon CE or Polyaxon Agent deployment
You can deploy/upgrade your Polyaxon CE or Polyaxon Agent deployment with access to data on the PVC.
Access to the dataset in your experiments/jobs
To expose the connection secret to one of the containers in your jobs or services:
run:
kind: job
connections: [dataset1]Or
run:
kind: job
connections: [dataset1, s3-dataset1]Use the initializer to load the dataset
To use the artifacts initializer to load the dataset
run:
kind: job
init:
- artifacts: {dirs: [...], files: [...]}
connection: "dataset1"