In SaaS your power user was free. In an AI product she's your largest bill.
Average AI product gross margin is 52% against the 80% that defined SaaS, and inference eats about 23% of revenue. Every growth instinct you inherited was formed in a world where your best user cost you nothing.
There's a rule most of us absorbed without ever being taught it: engagement is free. One more session, one more entry, one more day in the app — pure margin. That rule is why "north star metric" articles exist, why every dashboard puts DAU at the top, why nobody ever asked you what your heaviest cohort costs. It held for twenty years because the marginal cost of serving a software user really was approximately zero.
It doesn't hold for what you're building now, and the numbers on that are getting hard to argue with.
ICONIQ Capital's State of AI snapshot in January surveyed around 300 executives at companies building AI products. Average AI product gross margin: 52%, against the 70–80% that a mature SaaS business was expected to clear. At scaling-stage AI B2B companies, inference alone runs about 23% of revenue. Take the survey with the usual salt — self-reported, mostly B2B, published by an investor with a position in the outcome — but notice the direction before you dismiss it. That 52% is up from 41% in 2024. Margins are recovering, not collapsing. The headline everyone wrote about AI destroying software economics was directionally wrong; the interesting thing is subtler than that.
Here's the part that actually matters for a one-person product.
Your cost is now shaped like your usage, and your usage is not evenly distributed. It never was. You have a small tail of people who use the thing constantly, and in the old world that tail was your proof of product-market fit and cost you nothing to serve. In the new world that tail is your COGS. Your best cohort by retention can be your worst cohort by margin, and — this is the part that gets people — you almost certainly cannot see it. Your analytics tool reports engagement per user. Your model provider reports one number at the end of the month with no user attached to it. Nobody joins those two tables for you. So the flat $20 subscription keeps looking fine right up until the composition of your user base shifts.
Which brings me to the thing I'd push back on hardest, because it's the answer I hear whenever this comes up: costs are collapsing, so just wait.
The unit price genuinely is collapsing. GPT-4-class inference ran somewhere around $20–30 per million tokens in late 2022; equivalent performance is roughly $0.40–0.80 today. I'd treat the multipliers people quote with suspicion — I've seen 50x, 300x and 1,000x for the same three-year window, and which one you get depends entirely on which two models someone picked to compare. The direction is real. The round number is marketing.
But per-token price is only one half of your bill. The other half is tokens per task, and that is going the other way, fast. OpenRouter's own data has average prompt tokens per request up around 4x over thirteen months, with completion tokens nearly tripling. Reasoning models loop, re-check and reconsider; agentic flows make many calls where a chatbot made one. Comparisons floating around put a reasoning model's compute per task at tens of times a single-shot call on the same problem, and I don't fully trust any specific figure there, but the mechanism isn't in dispute.
So the price curve will not rescue you. Every saving it hands you, you will immediately spend — on reasoning, on longer context, on memory, on the multi-step version that finally works properly. That's not a failure of discipline. Better output costs more tokens. The savings arrive already earmarked.
Meanwhile the market is quietly repricing around this. Metronome's 2026 survey has 64% of SaaS companies billing at least partly by usage, up from 39% in 2024. A poll of 230 B2B software and AI companies found hybrid pricing — flat base plus a variable component — at 37%, up from 25% twelve months earlier. Both come from people who sell billing and pricing services, so read them as a direction rather than a measurement. The direction is that flat-rate is being abandoned by companies with finance teams, while indie makers are still shipping $20/month because it's the number that feels harmless.
What I'd actually do, and it's smaller than it sounds:
Compute cost per active user per month, not blended cost per request. The blended number hides exactly the thing you need to see.
Then split it by retention cohort. You are not looking for the average. You are looking for the margin on the users who love your product most, because those are the ones your growth plan is trying to make more of.
Before you ship a quality upgrade, price the new call first. "Make it smarter" is now a line item, and it's the easiest way to turn a working business into a broken one in a single release.
And if you do need a ceiling, design it into the product instead of bolting it on as a punishment. A natural stopping point — a session that ends, a reflection that concludes — reads as thoughtful. A hard cap that fires mid-use tells your most invested user she's used too much of the thing she's paying for. Same economics, completely different product.
The Murror version of this is uncomfortable and I'd rather say it than imply I'm above it. Our heaviest users are people going through something. The person journaling every day is in a hard month, and she is simultaneously the most expensive user to serve and the one I least want to meter. I don't have a clean answer to that. We made an unrelated decision last year to have the AI forget everything after thirty days, for trust reasons, and it happens to cap how much context every call carries — but I'd be rewriting history if I called that cost strategy. It wasn't. I got lucky, and getting lucky isn't a plan.
The honest summary is that the metric that used to mean "it's working" now means two things at once, and you need to be able to read both. Engagement is still the signal. It's just no longer free.
Replies
I went the other way on this: no subscription at all, just credits. One generation costs one credit, so the heaviest user is by definition the best-paying one and the per-user margin question mostly disappears.
The part nobody warned me about is refunds. Once each action has a visible price, a bad model response stops being a shrug and becomes a billing question. Inside a flashcard tool I build I ended up wiring refunds straight into the generation path: empty response, refund the credit; model delivers way under the requested count, refund. That forced me to actually price a single call, which flat-rate never made me do.
The trade I'd flag: credits move the metering anxiety from you to the user. People hoard them. Some sit on a full balance and never come back, which looks like churn but is really pricing friction. For Murror that trade feels wrong - someone journaling through a hard month shouldn't be watching a counter. Credits fit bursty, transactional tools. Your product is the opposite shape, which is why I think your "design the ceiling into the session" line is the right end of the stick for you.
@siarheihamanovich The refund path is the part I hadn't considered, and it's the best argument for credits I've heard. It forces you to know what one call costs. Flat-rate lets you avoid that question indefinitely — I've never priced a single Murror call, I price a month and hope.
The hoarding point is the one I'd want real data on. A full balance and no return looks identical to churn in every dashboard, but the fix is completely different: one is a product problem, the other is pricing friction. You could spend a quarter solving the wrong one.
And agreed on the shape. A counter visible during a journal entry changes what someone writes, which is the failure mode I care most about avoiding. Better the limit be the session ending than a number going down.
@monatruong_murror I don't have that data cleanly either, and I'm the one who built the thing that would produce it, which tells you somethin.
The cheapest probe I've found is the refill date. My free balance resets to 5 credits every 30 days, counted from each user's last renewal rather than a calendar date, so every free account hands me a small natural experiment on a staggered schedule. Come back within a day or 2 of the refill and the limit was the reason they stopped. Balance refills and nothing happens, that's the product. Same event two opposite readings, and it costs nothing to look at because the renewal timestamp is already sitting on the row.
Where it doesn't help is paid balances, which is exactly where hoarding lives. No refill event, nothing to read. Best I have there is the balance at the last session: if someone's final visit ended with credits still in the account, they didn't hit a wall, they just left.
@siarheihamanovich The staggered refill is the smartest cheap instrument I've seen for this, because the event is already on the row and it isn't aligned to anything else in the user's life. A calendar-date reset would collide with paydays and Mondays and you'd never untangle it.
One confound I'd watch: returning within a day of the refill also describes your highest-intent users, who'd have come back anyway. The way to separate those is to compare each account's gap-to-return against its own median gap before the balance ran out. If the refill return is much faster than that user's normal rhythm, the limit was the binding constraint. If it's the same, they were just due.
The paid-balance blind spot is the harder one and I don't think balance-at-last-session closes it. It rules out hitting a wall, but leaving with credits could be hoarding or could be that the last generation was bad and they quietly stopped trusting it. Your refund log might actually be the tiebreaker there - a refund in the final session or two reads very differently from a clean exit with money on the table.
We have a version of your staggered event, oddly. The 30-day memory wipe fires per user from their own start date, so I have a natural per-account boundary too. I've never looked at return behaviour around it. Adding that to the list.
True, your north-star metric can no longer be engagement alone. It has to be engagement × contribution margin.
@abrar_sami1 That's the right formula, and the hard part is that almost nobody can compute the second term. Engagement comes out of one tool; spend arrives from your model provider as a single monthly figure with no user attached to it. Until you join those two, you're multiplying by a variable you can't actually see.
Doing that join was more work than deciding to look, which is probably why most of us don't.
cost per active user split by retention cohort is the right instrument, but it quietly assumes cost and value land in the same month. i build Sorti, where the expensive call fires once at save time and the payoff shows up a median of 17 hours later, often more than a day. so the heavy cohort reads as the worst margin right up until you widen the window, and then it flips.
@linoy_bar_gal This is the objection I hadn't thought of and it's a real hole in what I wrote. Cost lands at save time, value lands 17 hours later, and a calendar month is an arbitrary bucket that can cut straight through the gap. Same problem as CAC payback, except at the level of a single call.
I'd push on one thing though: widening the window isn't quite the fix, because there's always a wider window that flips it back. The fix is setting the window from the lag you actually measured, so cost and payoff are guaranteed to be in the same bucket. And I'd want the distribution rather than the median - if the tail runs to a week, a 30-day cohort is fine, and if it runs to two months, no monthly view will ever read correctly.
Murror has the easy version of this. Cost and perceived value land in the same session, so my numbers can't lie to me the way yours can. That also means the instrument I recommended was tuned to my shape and I presented it as general, which is fair to call out.
The thing I'd be curious about at Sorti: does a user who never comes back to collect the payoff still cost you the same call? Because that's the cohort where the lag stops being an accounting artifact and becomes a genuinely bad unit.