Hello all!
We're Ben & Andreas, who made Replicate. Andreas used to do machine learning at Spotify. He built a lot of ML infrastructure there (versioning, training, deployment, etc). I used to be product manager for Docker's open source projects, and created Docker Compose.
We built https://www.arxiv-vanity.com/ together for fun, which led to us teaming up to build more tools for ML.
We spent a year talking to lots of people in the ML community and building all sorts of prototypes, but we kept on coming back to a foundational problem: not many people in machine learning use version control.
This causes all sorts of problems: people are manually keeping track of things in spreadsheets, model weights are scattered on S3, and results can’t be reproduced.
So why isn’t everyone using Git? Git doesn’t work well with machine learning. It can’t store trained machine learning models, it can’t handle key/value metadata, and it’s not designed to record information automatically from a training script. There are some solutions for these things, but they feel like band-aids.
We came to the conclusion that we need a native version control system for ML. It’s sufficiently different to normal software that we can’t just put band-aids on Git.
We'd love to hear your feedback. We're also trying to make this a community-built effort, so come say hi in Discord if you want to be involved in the early design and help build it: https://discord.gg/QmzJApGjyE
Report
Hunter
I caught this as an ex-colleague worked on it, who was talking about how hard it was to work on software before version control was free and readily available. This project seeks to solve that problem in the machine learning community.
Hugging Face
Zoo