Óscar Bustos

Robots.txt Generator - The right robots.txt file for your project

The robots.txt Generator is a powerful yet simple tool designed to help you create and manage your robots.txt file effortlessly. This tool lets you control how search engines, web crawlers or AI bots interact with your site.

Add a comment

Replies

Best
rupali kavale
Love the simple and quick nature of the product. Kudos for open sourcing it!
Benjamin Sloutsky
Congrats on the launch! Seems like a cool product.
Óscar Bustos
@benjamin_sloutsky Thanks Benjamin
Azra Memisevic
Wow, what a fantastic tool! It's great to see something that simplifies such a technical aspect of website management. Kudos to the team for making it open source and free for everyone!
Óscar Bustos
@azra_memisevic Thank you so much Azra
MV
I don't know if you're aware of it, but now you cannot select an item from the first list and an item from the second list (example if you want to have both wordpress default + blocking specific directories). A completely ignorant SEO person could hurt himself using this app, and an experienced SEO person doesn't need it. It's kinda of in the middle in my opinion.
Óscar Bustos
@markvenison Hi Mark, yeah I get your point of view will check if I can improve this with the UI. Thanks for share it and thanks for your support
Óscar Bustos
I wanted to thank everyone for the support I am receiving not only on Product Hunt but also on different social networks. Honestly, I didn't expect this simple tool to be so well-received. I am glad to know it can be helpful for people who dedicate time and effort to launching web projects. I would like to remind you that this tool is completely free and open source, so you can contribute to the project through GitHub. Best regards and thanks again.
Selly Park
Keeping this in my toolbox of amazing seo tools! 🧰 being an open source tool hope there’s more exciting updates coming up & would love to see the roadmap too! Congratulations on the launch! 🚀
Val
This tool seems like a fantastic solution for simplifying the management of robots.txt files. How does your tool handle complex scenarios where specific pages need different crawling rules based on user behavior or geographic location?
Raja Vijaya
Launched at the perfect moment = right when needed. Thank you
Nitin Joshi
I think you need to update it again. I cannot select both 'Based on your requirements' and 'Based on your project.' I want to allow a specific user agent to access my website along with the WordPress settings, but I cannot select it as such.
Moon
I love how Robots.txt Generator simplifies creating and managing robots.txt files, making it easy for all users without technical expertise to control their site interactions efficiently. Great job!