Crawler.sh
Free Local AEO & SEO Spider and a Markdown content extractor
450 followers
Free Local AEO & SEO Spider and a Markdown content extractor
450 followers
A fast, local-first web crawler and AEO & SEO analysis tool. Crawl entire sites in seconds from the terminal or a native desktop app, run automated SEO checks, extract content as clean Markdown, and export to JSON, CSV, or Sitemap XML.
What analysis is done for AEO? And what do you think plays crucial role to optimize sites for AEO?
Congratulations on the launch, @mehmetkose! Curious to see how the platform provides feedback on the issues
Love the local-first approach especially keeping everything fast and private
Curious — since it can crawl and extract entire sites, how are you handling safeguards around accidentally pulling sensitive or restricted data?
Feels like that could be important as usage scales
@mehmetkose Congrats on the launch! Doest it recommend actions & analysis report?
Great tool for SEO enthusiasts! How does Crawler.sh handle dynamic JavaScript-heavy websites during the crawl?
Triforce Todos
Fast, private, and outputs standard formats, that’s how tools should be. Congrats on the launch 🙌
Crawler.sh
@abod_rehman Thank you Abdul! It's written in Rust so, it's FAST. Just don't break anyone's website :)
Copperlane
Super cool - how does performance hold up on really large sites btw?