Featherless AI
Run every 🦙 model & more from 🤗 huggingface. Serverless
Start new thread
Wes George

Featherless AI - Run every 🦙 AI model & more from 🤗 huggingface

by•
Featherless is a platform to use the very latest open source AI models from Hugging Face. With hundreds of new models daily, you need dedicated tools to keep with the hype. No matter your use-case, find and use the state of the art AI model with Featherless.

Add a comment

Replies

Best
Eugene Cheah
Maker
📌
Hello Product Hunters! 🪶 I'm super excited to launch Featherless.AI today! A platform that allows quick access to all the top 🦙 models you see in Hugging Face 🤗 today. From the 8B to the merged 11B's and Qwen-2 72B's I know it's daunting for folks to download and set up all these large models on GPUs to try them, one at a time. Also renting GPUs can be incredibly expensive as it's typically several dollars per hour. On the flip side, popular providers may not have the various finetune, or even weird model sizes that you would love to try. That's why we built featherless AI. To eventually download and provide access to **all** of hugging face public models. Making all the various open-source AI models more accessible. Starting with a simple $10 or $25 monthly plan, with unlimited usage (within the concurrency limit). So anyone can use it personally, across any model, without worrying about token pricing in their day-to-day usage. Also: as the team who also helps build open-source foundation models (hey RWKV folks!), we fully understand the concerns various folks have over data security and privacy. As such, featherless.ai has no logging of any of your message prompts and completion. Why? Because we are not interested in stealing your data to train our models. So use it any way you want to, dun let someone else tell you how you should use your AI models. 🪄 The magic behind it? At the heart of it, is a custom-built inference infrastructure, built by the team here from Recursal.AI : which was built to be able to dynamically hot-swap models in sub-seconds. Allowing us to rapidly autoscale and dynamically adjust our infrastructure based on what models are popular. Once we have the models downloaded into our cluster. This allows us to provide more models, where previous providers have been limited in ensuring every model hosted has a dedicated GPU for it. 💬 In summary 🏃‍♂️ Run any of over 450+ huggingface models 🛠️ OpenAI compatible API, use your existing tools or client 📈 Unlimited usage (within concurrent usage) 🦙 Starting at $10/month for <15B models 🦅 To $25/month for 72B models 🎁 Special for PH: Signup with a subscription, and add referral `hello+producthunt@featherless.ai` for $10 off your next month bill Feel free to ask me anything here on the product hunt launch! And give it a try with a free trial, which allows you to chat with the models (up to a limited amount of messages) at https://featherless.ai
Jonathan Leung
Dooooope @eugene_cheah
Eugene Cheah
Maker
@manisha_hr_ Thanks! Its OpenAI API - so if your devs were already using OpenAI styled API for AI, they should be able to try this instantly (as its the same API)
Eugene Cheah
Maker
Eugene Cheah
Maker
@amit44 Thanks!
Shi Ling
Awesome. It's nice to be able to quickly try out and preview different llama models without having to deploy them on my own servers. Been meaning to the different variety of roleplay models for my dnd sessions. Are you planning to support other open-source models besides llama?
Eugene Cheah
Maker
@taishiling - im glad you like being able to play with all the models. Yes, we are currently downloading more models which would be coming online over the next few days. We currently support RWKV and LLaMA based models. We plan to introduce Mistral MoE's next (to be confirmed), followed by potentially larger models. The main reason we started with llama, is because it was the largest pool of all the popular models our initial users and community wanted to use. But the goal remains: ALL huggingface models. One major group at a time
Julien Chaumond
wow this is very cool! @eugene_cheah
Eugene Cheah
Maker
@julien_c Thank you 🙏 : now we just need to scale usage, then servers, then models - until we catch them all from huggingface 😉
Julien Chaumond
@eugene_cheah let's do it! BTW https://recursal.ai/ looks quite cool too
Eugene Cheah
Maker
@julien_c Thanks! They are actually the same backend / infra 😉 As we scale featherless.ai, we do plan to integrate it back to recursal, with the more "complicated features" like automated finetuning, etc. So that recursal will be providing offerings beyond RWKV.
Kyrylo Silin
Hi Eugene, Featherless.AI sounds like an invaluable tool for accessing and experimenting with various LLMs. How do you manage the computational resources to support such a wide range of models? Also, are there plans to support custom model training or fine-tuning on the platform? Congrats!
Eugene Cheah
Maker
@kyrylosilin Our infrastructure auto scale according to current workload, with optimizations specific to our infra provider. We also adjust our setup for the best price to performance GPUs, to lower overall cost (aka no H100s), even for the larger models. In overall, this lets us keep costs to the minimum (no wasted GPUs), according to the user workloads. And yes, we do plan to support additional features like finetuning, and the option to switch to token based pricing, for larger scale commercial users in the future. But for now, this is aimed at individuals who would love to play with all the models
Inoticed your pricing details did not mention anything about API access. Perhaps you could update it so non PH users can see it too :) Congrats on the launch!
Eugene Cheah
Maker
@charlestehio Thats a good point, will make it more obvious! Thanks!
William Bowen
Great product, very useful to be able to use and change llama models instantly, without having to set up and deploy a server myself!!
Eugene Cheah
Maker
@william_bowen4 Exactly! Useful when you want to test many models!
Nathan Wilce
Congrats on the launch: I was wondering how this is feasible with the cost of gpus and hosting this many models?
Eugene Cheah
Maker
@nathan_wilce Thanks! All the models are pre-downloaded to our clusters, and are on standby. The GPUs spin's up and swap these models on the fly under seconds. This was made possible with our custom built inference systems we built and optimized on. Allowing us to keep cost low, and auto scale to actual usage. PS: 450+ models takes up over 9TB of storage space, we are downloading 100+ more which will be coming online soon. (it takes time haha)
Andreas Sohns
Congrats on the launch! Featherless AI offers great access to Hugging Face models at a fair price Excited to see what it can do!
Eugene Cheah
Maker
@andreas_sohns Thanks! Next step: More models! After that: To make it easy for anyone to finetune their own models, for their use case. And use it =)
Hossein Yazdi
This is very handy, great one Eugene!
Eugene Cheah
Maker
@hosseinyazdi - Thank you 🙏
Tim Pouw
Interesting. Shared it with our dev lead!
Eugene Cheah
Maker
@tim__pouw Thanks, hope you can find a useful model for your use case, or for fun!
12
Next
Last