We received a lot of feedback from our users, and this helped us prioritize our work. In this blog post I will go over some of the new features that we made available since Polyaxon 0.1.
First things first, documentation!
We received a lot of feedback regarding our documentation and deployment workflow, I am sure that there’s still a lot of work to do, but we made some improvement:
We created a first version of our deployment app, it’s still under heavy work and once it’s finished it will replace the install section in the docs.
We made better documentation for our client, created a new section for the REST API and release notes.
Query filters and sorting
Having a platform that provides a scalable hyperparameters tuning algorithms made it possible for some of our users to explore very large search spaces to optimize their ML/DL models. However, we notice a problem, there was no simple way to explore the results of the experiments. Users needed to paginate through hundreds if not thousands of experiments per projects. This was not optimal, and so we made a new query api with a syntax allowing to filter and sort the experiments and jobs.
This query syntax can be used by the CLI:
polyaxon project experiments -q 'status: succeeded|failed, metric.loss:<=0.3'And in the dashboard:
This was quite important to find out quickly the results of the experiments, and save important searches for future visits to the dashboard.
This Query api allows to search basically by metrics, params, builds, commits, users, and many different meta data that Polyaxon collects during the training phase.
Dashboard
The dashboard was basically showing list of experiments, and was far behind our CLI in terms of functionality. So we made a lot of progress on that.
We just mentioned the search functionality, that was one, but we also introduced some new functionalities, annotations, and interactions.
We also exposed some data and events that the platform collects, to provide better compliance for our users:
- Activity logs
- Recently viewed
- Bookmarks
Visualization
We received an overwhelming feedback from our users to have visualization of the metrics tracked by Polyaxon, so we made some basic visualisations in the dashboard. Polyaxon 0.3 will have more customizable visualizations, experiments comparison, and model analysis.
Integrations
Running large number of experiments for long durations requires notifications, so that users can focus on other tasks while their jobs are running on the cluster, and only get notified if their experiments required their attention.
We also provided easier ways to onboard team members after deploying the platform through single sign on.
Single Sign On with gitlab, bitbucket, github, Azure, and LDAP.
Notifications with, email, slack, mattermost, pagerduty, discord, …
Support of private registries, and external postgresql databases.
Scaling
For larger teams or teams with bigger clusters, we made it very easy to scale the platform, there’s a lot of ways to scale Polyaxon; replicating the API and the workers is one of the easiest, so we made sure that you can scale polyaxon to run tens of thousands of experiments concurrently.
We also removed the restriction of only mounting one persistent volume for data and outputs at a time, now users can mount multiple volumes per experiments in an easy way, just by referring to the name of the volumes configured during the deployment.
We also worked on storage and persistence abstraction that will be available soon, it will allow users to not only mount k8s volumes but also use S3, GCS, and Azure storage for their jobs and experiments in a seamless way.
Conclusion
This was only a couple of improvements among many that we worked on during this version. Next version we are focusing on on better tracking, visualizations, model comparison, and model analysis. We also want to make it easier to use cloud platforms, so you should expect better integrations for storage as well as other aspects. Finally, we want to allow users to use github/gitlab/bitbucket as the source of truth, in case they don’t want Polyaxon to handle code tracking.
Polyaxon will keep helping data science teams to be more productive and iterate faster on their machine learning and deep learning models. We are also very open to suggestions, feedback, and criticism. Polyaxon is open source, and we would like to involve a bigger community to solve issues related to reproducibility in data science.
