noor abid

I ve been testing a safety check for an AI app by running a script of test messages many times against the model. Every run was fine until it wasn t: after a day of repeated runs, calls started failing with 429 errors, and waiting 60 seconds between retries stopped helping.

I assumed the limit was per minute. It wasn t. On the free plan for the model I use, the limits are 30 requests a minute, 1,000 requests a day, 8,000 tokens a minute and 200,000 tokens a day. I believe I hit the daily cap, because the app itself also stopped generating responses at the same time. My tests and the app share one account, so they share one budget.

What I m changing: run tests in small batches, never repeat full runs, and plan a backup provider so a quota problem doesn t become an outage.

How do you keep testing from eating your production quota? Separate accounts, a cheaper model for tests, or something else?

Ember tracks five categories (sleep, fitness, discipline, focus, spending). My code picks which one gets asked next. My code compares your new answer to your last one and works out how the grade changed. My code decides when an achievement unlocks. The model gets those results and writes the roast around them, and it s only allowed to quote numbers that actually changed. For example, if your sleep grade went from C to B, that s the fact it gets. It doesn t get to guess.

I m not claiming this makes it perfect. But when a roast looks wrong, I know where to look: is the data the model received wrong (my bug), or did it write badly from correct data?

One related bug I hit: achievements kept un-unlocking themselves because three parts of the code wrote to the same row without checking each other. The fix was a compare-and-swap write, so the write succeeds only if the row is still what it was expected to be.

Where do you draw the line between what the model decides and what plain code decides?

Hi PH, Noor here, solo founder of Ember (emberai.site). Ember runs on Groq's gpt-oss-120b. My first rate-limit errors (HTTP 429) didn't come from users. They came from running my own test suite. So before I market anything, I did the math. Everything below is my estimate, not measured on production.

The numbers: Groq lists this model at about $0.15 per million input tokens and $0.60 per million output tokens (from third-party pages; I'm verifying in my console). A first roast uses roughly 6,000 tokens. A signup that also generates a plan is roughly 15,000. A check-in makes 5 sequential model calls, about 8,000 to 10,000 tokens. That works out to about half a cent per full signup, and roughly 10 cents a month for a free user who checks in daily.

Where it actually breaks: not on money, on speed limits. I'm on Groq's free plan today: about 30 requests and 8,000 tokens a minute, which is less than one signup's worth every two minutes. The paid developer tier is reported at around 250,000 to 300,000 tokens a minute, about 17 signups a minute. So 1,000 people signing up in the same hour would need about 15 million tokens, roughly an hour of queueing.

What I'm doing about it: upgrade before marketing, set a spending cap, make the model name swappable (it's copied into 5 files right now), and plan a second provider. The same model is listed cheaper on other hosts, so a backup could also save money.