Humanloop

Name: Humanloop
Rating: 5.0 (1 reviews)

Humanloop is the LLM evals platform for enterprises

5.0•1 review•

93 followers

Humanloop is the LLM evals platform for enterprises

5.0•1 review•

93 followers

Visit website

AI Infrastructure Tools

•

AI Metrics and Evaluation

Humanloop is the LLM evals platform for enterprises. Teams at Gusto, Vanta and Duolingo use Humanloop to ship reliable AI products. We enable you to adopt best practices for prompt management, evaluation and observability.

Launch tags:Developer Tools•Artificial Intelligence

Launch Team

Customer.ioAutomate Messaging Everywhere — Startups Get 12 Months Free

Promoted

Squarecraft.io

Congrats on the launch! Out of curiosity, what NLP models are you training? Is it like a pretrained BERT with a classification head?

Report

6yr ago

Memorai 🧠

Maker

@aveni Hey Abhinav, we've actually wrapped a few of the major NLP libraries so we can choose between quite a lot of models. By default its a pre-trained distilbert model but we also have lighter models (lstm/gru + linear layer) and more powerful models (full bert etc) At the moment you get a fixed default model and we work with customers to set the most appropriate option but we're working hard behind the scenes to implement automatic model selection.

Report

6yr ago

JSON Web Token (JWT) Debugger

Very cool. Do you have plans to create integrations with databases/data sets? Very cool user experience.

Report

6yr ago

Memorai 🧠

Maker

@robert_thelen We're building out connections in response to user demand. If you have a specific use case in mind please ping me and I'd be happy to discuss

Report

6yr ago

Memorai 🧠

Maker

Hi everyone! Peter, Jordan and Raza here. We're really excited to share what we've been building over the last few months and get your feedback. 😅 We're a team of AI engineers and researchers from University College London who have worked on AI projects at Google, Amazon and Microsoft. We built Humanloop to make the breakthroughs in natural language understanding we were seeing in the lab, accessible to everyone. Getting NLP models into production still requires annotating a huge number of examples and a lot of specialist expertise. The Humanloop platform takes care of model training, data selection and deployment for you. You upload your data and we provide an annotation interface for you to teach a classifier. As you label we train a model, work out what data is most valuable and then deploy the model for you. Some of our early customers are using Humanloop to automatically route support tickets, understand twitter data and moderate user generated content. We use new techniques in active learning to dramatically reduce how much labelled data is needed to train language AI. This removes one of the biggest costs and barriers for teams wanting to build their own machine learning models. We'd love to hear your thoughts and ideas for how you might use your own natural language AI!

Report

6yr ago

HireSweet

Interesting! Do you support the ability to manage teams of annotators?

Report

6yr ago

Memorai 🧠

Maker

@robin_choy Hi Robin! Yes we do but not on the free tier. On a teams account, all you need to do is input their list of email addresses and it will invite them as annotators to the project. All the annotators then annotate against the same model. Do you have a specific project in mind?

Report

6yr ago

Just been playing around with it, it's awesome! Really nice to use

Report

6yr ago

Memorai 🧠

Maker

@waichuen glad you liked it! Any ideas for what you might want to build with it?

Report

6yr ago

@raza_habib We do a fair amount of NER so if we could annotate individual spans of text, we'd be all over this! We've had to make our own internal labelling tool, but it's way less slick and doesn't have predictions built in

Report

6yr ago

Memorai 🧠

Maker

@raza_habib @waichuen That's great to hear! We are building the sequence level annotation functionality right now, I'll send you an email. R

Report

6yr ago