What’s the hardest part of web scraping in production today?

I’m curious — for teams running scraping or data extraction in production, what breaks the most?

Is it:

• JS-heavy websites?

• Frequent DOM changes?

• Anti-bot protection?

• LLM cost & hallucinations?

• Keeping extracted data consistent over time?

I’m building LucidExtractor after facing many of these issues myself, and I’d love to learn how others are handling this in the real world.