How much trust do you lose when your app displays wrong scraped data?

by•

Parsing messy public web data is honestly harder than building the actual core feature.

You think a website's meta tags will have standard company info and then you find sites that put random blog text in the title tag. If your tool makes a wild guess and gets it wrong the user instantly loses trust in the whole product. I've started marking scraped fields as "needs review" whenever the confidence score drops below a certain point.

Have you found a reliable way to clean up dirty data before it hits the user interface?

5 views

Add a comment

Replies

Be the first to reply

Have a question or a thought to share? Add a comment above to start the conversation.