When do AI API costs become too high to justify the convenience?

API costs are manageable when an app is small but as usage grows the monthly bill can get hard to ignore. I've been wondering if switching to open-source models like Llama or Mistral would actually save money. Running your own setup might lower costs but GPU bills maintenance and model quality are still concerns. For those who've tried both did self-hosting really save money or was sticking with APIs worth it?