What happens when an AI agent optimizes a cost your simulator doesn’t model?

Something we’ve been thinking about while building Cloud World Model:

If an infrastructure simulator doesn’t model a particular cost, a human might recognize that the estimate is incomplete.

An RL agent may not.

To the agent, an unmodeled cost can effectively look like $0 - which means the agent can learn to exploit the gap and recommend an architecture that only appears cheaper because the simulator doesn’t understand part of the bill.

We decided CWM shouldn’t hide that uncertainty. Simulation results now distinguish between what’s modeled, estimated, and a known gap, rather than pretending every number has equal confidence.

I’m curious how others building agentic systems think about this:

Should agents be allowed to optimize across partially modeled environments? Or should insufficient coverage automatically constrain the actions they’re allowed to take?