How we handle data sovereignty and compliance for on-premise AI agent deployment

Specific because vague compliance talk helps nobody

The architecture we use for fully on-prem AI agent deployments:

- Local Llama 3 70B as the primary reasoning model, runs on customer hardware

- All prompt and response data stays inside the customer's VPC, never leaves

- Air-gap mode supported where outbound network access is fully blocked

- Audit logs written to customer's existing SIEM (Splunk, Datadog) via local collector

- Model updates delivered via signed offline packages, not network pull

The hardest part wasn't the model. It was the operational tooling around updates and observability when you can't phone home.

What's the on-prem architecture decision you've made that surprised you in deployment?