AI Governance Is Becoming An Infrastructure Problem
Most organizations have written their AI governance policy. A document that defines acceptable use, specifies prohibited applications, and nominates a responsible owner. In theory, governance is handled. In practice, the policy document has no relationship to what the models are actually doing. The hidden tension is that AI governance written as policy operates at the wrong layer of the stack. A…
- Governance that can't be audited can't be enforced, logging is the foundation of operational AI governance.
- Model cards describe models; model registries govern deployments, they're not interchangeable.
- Access control for model endpoints needs to be model-aware and capability-scoped, not just request-authenticated.
- Deployment approval gates enforce governance before production; reviews only tell you what went wrong after.
- Retrofitting governance into a deployed AI system costs an order of magnitude more than building it in from the start.
Policy Documents Cannot Enforce Themselves at Inference Time
A governance policy is a statement of intent. Infrastructure is what enforces it. The organizations that have learned this distinction the hard way are the ones that deployed models under a governance framework and then discovered their framework had no hooks into the systems where decisions were actually being made. The specific failure pattern: a policy document prohibits use of the model for certain categories of decision. No mechanism exists to detect when those decisions are being made. No
Model Cards Are Not Governance Artifacts
Model cards describe what a model is. They don't track how a model is being used, when its behavior profile has changed, which version is deployed in which environment, or who approved the deployment. Model cards are documentation. Governance requires a model registry. A model registry is the operational layer that turns model metadata into enforced governance. It tracks version lineage, which fine-tuned version was derived from which base model, trained on which dataset. It records approval st
Inference Audit Logging Needs to Be Designed Before Deployment
Retrofitting audit logging into a deployed inference system is technically possible and operationally painful. The schema decisions made during initial deployment, what to log, at what granularity, in what format, shape what's available for compliance review, incident investigation, and governance reporting for the lifetime of the deployment. The minimum viable inference audit log captures: request timestamp, requesting identity (user or service), model identifier and version, input hash (not f
Frequently asked questions
- What's the difference between AI governance and AI compliance?
- Compliance is meeting external requirements, regulatory mandates, audit standards, contractual obligations. Governance is the internal framework that makes compliance sustainable. You can pass a compliance audit with a policy document and no infrastructure. You cannot sustain compliance across a growing AI deployment without infrastructure that en…
- Do we need a model registry if we only use models from one vendor?
- Yes. A model registry isn't primarily about tracking models from multiple sources, it's about tracking versions, deployment states, and approval history for whatever models you operate. Even if all your models come from a single provider, you still need to know which version is deployed where, who approved each deployment, and what the audit trail…
- How do you make inference audit logging privacy-compliant?
- The core technique is separating the governance log from the content log. The governance log, timestamps, identities, model versions, request hashes, is retained for audit purposes with standard access controls. Full request and response content, if retained at all, is stored in encrypted cold storage with strict access controls and defined retent…
- How do deployment approval workflows scale without becoming a bottleneck?
- The answer is risk-tiering. Not all model deployments require the same approval depth. An internal developer tool with no sensitive data access and no customer-facing output might need engineering and security sign-off. A customer-facing deployment processing financial data might need legal and privacy review as well. The governance framework defi…