Polyaxon & Data on Ceph

How to use Polyaxon and Data on Ceph together

About Ceph

Ceph is an open-source software-defined storage platform that implements object storage on a single distributed computer cluster and provides 3-in-1 interfaces for object-, block- and file-level storage. Ceph aims primarily for completely distributed operation without a single point of failure, scalability to the exabyte level, and to be freely available. Since version 12 Ceph does not rely on other filesystems and can directly manage HDDs and SSDs with its own storage backend BlueStore and can completely self reliantly expose a POSIX filesystem.

Integrations/data-stores

You can use one or multiple buckets on Ceph to access data directly on your machine learning experiments and jobs.

Deploy Ceph

Before using Ceph, you need to deploy it and create a bucket (or several) to host your data.

Create a bucket on Ceph

You should create a bucket (e.g. plx-storage) where you will host your data.

Use the bucket in Polyaxon

In order to use the buckets with Polyaxon, you can follow the S3 DataStore integration.

Please note that you will need to set as well AWS_ENDPOINT_URL or S3_ENDPOINT_URL.

Version

This is the docs of the latest stable version.

Improve this page!

If you spot a typo or have a suggestion to make this page better, please make a contribution to improve the documentation.

Update this page

Have a feedback?

We do our best to make this documentation clear and user friendly, if you see anything that is incorrect or have any questions, feel free to reach out!

Send Feedback