Three quarters of the clicks past our bot flag were one crawl
Up front, I build and run affiliate sites for other people as a paid service, so a click count is something I hand to someone else, and I am not neutral about it being right.
We log every outbound click ourselves instead of reading the network report back, one row per click carrying product, address, signature and time. That decision is what made the rest visible. On one UK tag the report handed me a count of 1,375 and nothing underneath it.
On the site I run myself that table holds 16,976 rows. The flag marked 11,155 of them and left 5,821. Bucketed by day, 4,336 of the survivors sit inside one three day window in August, reaching 674 products from 768 addresses. That is a crawl, and no total would ever have said so.
So the flag is not the number. How clicks fall across days, and how many addresses touched how many products, is what I read before quoting a total, and that is what the row level table is for.
If you keep your own table, what makes a row trustworthy to you?
Replies
My opinion is the flag is the wrong tool. A row only becomes trustworthy when I can rebuild the session behind it. One address touching 674 products in three days was never a visitor.
@maxwell_wx Agreed on rebuilding the session, and one correction on the shape, since it makes your point stronger rather than weaker. It was not one address reaching 674 products. It was 768 addresses reaching 674 products inside those three days, about six hits each, which is why per-address rate limiting would have caught none of it. Spread across enough addresses, every individual one looks like a light browsing session. That is the case where the session view and the flag both pass and the total is still wrong. What survived it here was time. Arrivals clustered into one three day window against a table that spans months, and 226 repeats of a single signature among rows already cleared. Neither of those lives inside one session, which is the argument for keeping the raw rows rather than a verdict written at collection time. If you rebuild sessions, do you keep the raw rows after, or does the session become the record?