Agent Engineering Lab · Guide
AI Governance, from Scratch
How to stand up AI governance in an organisation that has none, and make it the reason your teams can move fast. One path, twenty-two steps, in this order.
Start at Step 1 See all 22 steps
7 of 22 steps are published. They set your purpose, mandate, policy, register and risk tiers, and the habits that get them followed. Part III, Setting it up is next. New steps appear in the RSS feed.
How to use this guide
- Start at Step 1. Each step opens with what done looks like. If you can show it, move on. If you can only describe it, you are not done.
- Read in order. Each step rests on the ones before it. When you apply it, take one real system through every step as you go.
- Don't skip Part II. It is the reason the rest gets followed instead of routed around.
- Deep dives can wait. The top half of a step gets it working. The deep dive is for the edge cases you meet later.
- Adapt the defaults. They are strict, and written for regulated industries. Set tiers, decision rights and evidence to your own risk appetite and the rules you answer to.
- Extend what you have. If security, privacy, procurement, model risk or change management already work, add AI to them. Build nothing in parallel.
The path
Twenty-two steps in six parts. Each one tells you what done looks like.
Part IFoundations
Published- 1 Agree what governance is for Done when Your sponsor can explain, in a minute, why it makes delivery faster. Read
- 2 Get a mandate and a one-page AI policy Done when An executive owns AI risk, and the policy is published. Read
- 3 List every AI system
Enterprise
Done when Nothing reaches production without a register entry.ReadSmall team
Done when Every system is listed in one file in a repo, and no model credential is issued without an entry. - 4 Tier every system, and say no to some
Enterprise
Done when Each entry has a tier, and the tier changes what it must pass.ReadSmall team
Done when Each entry has a tier, and the tier decides who signs it off and what it must pass.
Part IIPeople before process
Published- 5 Find where your engineers will route around you Done when You can name the three shortcuts your teams take most. Read
- 6 Make the governed path the fastest path Done when The paved road is quicker than the workaround. Read
- 7 Build a culture that reports problems Done when Someone reported a near-miss and was thanked for it. Read
Part IIISetting it up
Coming- 8 Name owners and decision rights Done when Every decision has one accountable person. Coming
- 9 Put gates between idea and production Done when A system can't deploy without its tier's evidence pack. Coming
- 10 Make evidence block releases Done when A failing evaluation has stopped a release. Coming
- 11 Run it: watch, respond, review, retire Done when An incident drill has run end to end. Coming
Part IVResponsible AI in depth
Coming- 12 Turn principles into tests Done when Every principle in your policy has a test and an owner. Coming
- 13 Choose your fairness trade-off on purpose Done when The fairness metric was chosen, written down and signed. Coming
- 14 Explain, disclose and allow appeal Done when An affected person can find out, and contest the outcome. Coming
- 15 Make human oversight real Done when Reviewers can, and sometimes do, overrule the system. Coming
- 16 Bound reliability, safety and privacy Done when Hallucination, abuse and leak limits are tested per tier. Coming
Part VGenAI and agents
Coming- 17 Govern models, vendors, prompts and retrieval Done when Prompts and model versions are reviewed and versioned like code. Coming
- 18 Give agents identity and bounded authority Done when No agent can call a tool it wasn't granted. Coming
Part VIEnterprise scale
Coming- 19 Federate: central standards, local delivery Done when Business units run governance without queuing on the centre. Coming
- 20 Build governance into the platform Done when Most controls run automatically on the paved road. Coming
- 21 Reuse one set of controls and evidence across regulators Done when One evidence pack supports every obligation that applies to you, with the gaps written down. Coming
CloseProve it on one system
Coming- 22 Walk one system through it all Done when You can trace one high-risk system through every step and show the evidence at each. Coming
The guide on one sheet
Each step's call, and the files to start from. All 22 are here, including the steps still to be written up. The files come from the open AI governance framework, pinned at v2.2.0.
| Step | The call | Start from |
|---|---|---|
| 1 Purpose | Governance exists to make safe delivery faster. | Framework overview, Start here |
| 2 Mandate and policy | One page, one executive owner. A policy nobody can test is a wish. | AI policy, Acceptable use, AI risk committee charter |
| 3 Register | The register is the gate: no entry, no production credential. | AI inventory entry, AI inventory |
| 4 Tier | Govern by tier, not by project. Decide the prohibited list before the first request. | Risk matrix, GenAI risk matrix, When to say no, GenAI use case assessment |
| 5 Known shortcuts | A shortcut tells you something about your process. | Acceptable use, Governance metrics, Start here |
| 6 Paved road | Make the safe way the easy way, and enforce it in the pipeline, not in a meeting. | Reference implementations overview, Customer service chatbot gate input |
| 7 Reporting culture | Thank people for near-misses. A culture that hides problems defeats every control. | Incident forensics, Incident report GenAI |
| 8 Owners | Builders never validate their own high-risk systems. | Roles responsibilities, Roles GenAI, Escalation paths |
| 9 Gates | A gate that can't stop a release isn't a gate. | Governance workflow, Deployment gates, Control register |
| 10 Evidence that blocks | A gate that has never said no has not been tested. | Evaluation governance, Red teaming protocol, Red team report |
| 11 Run and respond | Every exception expires. Every system has a way to stop. | Monitoring, Incident report template, Exception register, Review cadence |
| 12 Tests | A principle without a test is a slogan. | Responsible AI checklist, Impact assessment template |
| 13 Fairness | You cannot satisfy every fairness metric at once, so choose one in the open. | Bias detection LLM |
| 14 Explain · appeal | If a person cannot contest it, you have not explained it. | Transparency standards, Model card template |
| 15 Oversight | A human who never overrules is a rubber stamp, not oversight. | Human in loop patterns |
| 16 Safety · privacy | Set the limit before you measure, not after. | Hallucination policy, Adversarial robustness |
| 17 Models · prompts · RAG | A prompt change is a code change. | Model selection, Prompt engineering standards, RAG governance, Third party model risk |
| 18 Agent identity · authority | Agents get identity and least authority before they get tools. | Agent identity and tool governance, Multi agent governance, Agentic AI risk |
| 19 Same stack, every business unit | The centre sets standards and builds the road. Teams deliver. | AI CoE design, Governance metrics, Board reporting, AI literacy |
| 20 Platform | Automate the controls that repeat. Keep committees for high-risk approvals and exceptions. | Reference implementations overview, Deployment controls, GRC integration |
| 21 One evidence pack, reused across regulators | Keep one mapping from rules to controls and reuse it. Never build a programme per regulator. | EU AI Act mapping, ISO/IEC 42001 mapping, NIST AI RMF mapping, Multi jurisdictional guide, Regulatory change monitoring |
| 22 Walk-through | Governance is proven on one real system, not on a slide. | Customer service chatbot case file |
What each tier demands
Step 4 gives every system a tier from T1, the highest risk, to T4. The tier then sets who approves it, who checks it and how often it is looked at again.
| T1 · Critical | T2 · High | T3 · Medium | T4 · Low | |
|---|---|---|---|---|
| Approves the risk assessment | AI Risk Committee | AI Risk (second line) | Team lead | Team lead (self-assessment acknowledged) |
| Independent validation | MRM (full scope) | MRM or another second-line reviewer outside the development team (reduced scope) | Peer review | Self-test |
| Red-team depth (GenAI systems) | 200+, including manual testing (external testers recommended) | 100+ | Automated adversarial scan + 50 manual cases | Automated scan, 20+ cases (recommended) |
| Metrics review | Daily dashboard check by AI Operations, plus a monthly system-owner review | Weekly | Monthly | Quarterly |
| Formal governance review | Quarterly | Semi-annual | Annual | Annual self-assessment |
| Revalidation | Annual, and on material change | Every 24 months, and on material change | On material change (peer re-review) | On material change (peer re-review) |
How it fits together
Everything you answer to keeps growing above. Every team, system and agent keeps growing below. Wire each rule to each team and the connections multiply. Put one narrow waist between them and they add.
- EVERYTHING YOU ANSWER TOEU AI Act · ISO/IEC 42001 · NIST AI RMF · SR 26-2 · board · customers
- Principles and policy, mapped to one control setSteps 2 · 12 to 16 · 21
- THE NARROW WAISTregister · tier · controls
evidence pack · platformSteps 3 · 4 · 9 · 10 · 20 - Paved road, owners and gates per teamSteps 5 to 11 · 19
- EVERY TEAM · SYSTEM · AGENT · BUSINESS UNITSteps 17 · 18 · and whatever you build next
What each step rests on
Read it from the bottom up: foundations at the base, scale at the top. Select a step to see what it rests on, what rests on it, and the patterns it puts into play.
Brick outline and tint: the selected step. Solid line: what it rests on. Dashed outline: what rests on it. Pattern colours follow the catalogue: blue capability, brick control, gold evidence; a diamond marks an anti-pattern.