We gave AI 108 real ads and asked it to think like an advertiser. Here's what happened.

We had a dumb but nagging question.

Everyone's racing to use AI for advertising. But does AI actually understand an ad? Like, really understand it — the message, who it's talking to, whether it's landing? Or is it just vibing and generating plausible-sounding words?

We couldn't find a benchmark that tested this. So we built one.

We took 108 real ads — the kind you'd scroll past on Instagram or see on a billboard in Tokyo. Fashion, fintech, food, health, FMCG, D2C. USA, EU, Asia. Some with influencers, some without. The kind of messy, real-world creative that agencies actually deal with.

Then we asked 4 AI models to do something simple: look at each ad and tell us — what's the message, who's it for, is it working, and how would you fix it?

What surprised us most wasn't the scores. It was where the models fell apart.

General-purpose LLMs were fine at describing what was in the ad. But ask them "who is this really talking to?" on a hyper-local campaign for a regional food brand in Southeast Asia — and they'd give you a beautifully confident, completely generic answer. They'd miss the cultural context entirely. Like asking someone who's never left their hometown to review a neighbourhood restaurant guide.

The other thing that broke them: actionable feedback. The suggestions were technically fine, just... useless in practice. Think "improve the visual hierarchy" energy. Thanks. Very helpful.

A few things we genuinely didn't expect:

• Influencer ads were harder than we thought — not because of the person, but because the model had to understand the relationship between the creator and the product claim

• Local vs. global scope created a bigger performance gap than vertical or category

• One model was confidently wrong in a way that would've cost real money if someone acted on it

We're sharing the full paper not because the results are a victory lap, but because we think the advertising industry needs to start asking harder questions about what "AI understands your creative" actually means.

Curious if others have run into this — moments where an AI gave you ad feedback that sounded smart but felt completely off? Would love to compare notes.

Full paper here: blog.ad-vertly.ai/how-well-ai-can-reason-over-real-world-ads/