🧠 Which decisions in your product actually need a big model?

by•

Last week we timed one small decision in our app:

picking which part of the system should handle a message. 2.4 seconds, on a model that can write essays, to answer a multiple-choice question.

It made me look at every model call differently. Some need to write. Most just need to choose, score, or check.

Where are you using a big model for something a small, fast one could decide? And has swapping one ever bitten you?

8 views

Add a comment

Replies

Be the first to reply

Have a question or a thought to share? Add a comment above to start the conversation.