The safety problem with AI agents isn't the model.
by•
Every agent framework asks the model to assess its own actions — a confidence score, a risk level, a sentence about why something is fine to proceed with. That's the wrong shape. A model that can tell you an action is safe can be persuaded to tell you an action is safe, and nothing downstream can tell the two apart.
So I built the permission layer so the model's judgment only moves one direction. A registry sets the floor. The model can raise a requirement above it and can never lower one. It can flag, it cannot reassure. What's interesting is that this gets stronger as models get smarter, not weaker — a better model flags better and still can't wave anything through.
1 view


Replies