Generative AI governance
Most AI governance was written for systems that score or classify. Generative systems produce something new every time, and that changes where the control has to sit.
Generative AI governance comes down to three decisions you have to make deliberately: what information may go in, who checks what comes out, and which of your existing duties already apply to it.
Most governance documents were drafted with a different kind of AI in mind: something that scores an application or sorts a queue. A lot of that thinking still transfers. What does not transfer is the assumption that you can check the output against a right answer.
What actually changes?
The output is new work rather than a decision, and usually there is nothing to test it against. A credit model either approved the application or it did not, and you can audit that against the rules it was meant to apply. A drafted client email is either something the partner would have sent or something they would not, and only a person who knows the client can tell you which.
So the control moves. With a scoring system you govern the model, its data and its thresholds. With a generative one, most of the governance lands on two human decisions instead: what people are allowed to put in, and who looks at what comes out before it goes anywhere. That is a smaller technical problem and a much larger organisational one.
The output can be wrong and still read as right
This is the risk people underestimate, because a fabricated answer arrives in the same confident register as a correct one. The the International AI Safety Report 2026 is direct about the state of it: models still produce hallucinations, particularly on multi-step tasks and on anything requiring reasoning about the physical world, and current mitigation techniques can reduce failure rates but not to the level many high-stakes settings need.
The same report notes that systems acting autonomously carry more of this risk, not less, because there are fewer points where a person would otherwise notice. If you are letting a tool take actions rather than draft suggestions, the review you removed is the control you were relying on.
None of that is an argument against using these tools. It is an argument for knowing which pieces of work can absorb an error and which cannot, and putting a person in front of the second kind. The the NIST Generative AI Profile sets out governance and incident-handling structure specific to generative systems if you want a shape to borrow.
What is allowed to go in?
The question staff actually ask is whether they can paste a client document into a chat window, and they ask it once, silently, and then decide for themselves. Whatever you write down, this is the decision that gets made most often and observed least.
Two things make it manageable. The first is naming approved tools rather than approved behaviour, because "use good judgement with confidential information" gives nobody a way to act. The second is being specific about categories: personal data, client-confidential material, anything under an NDA, unreleased financials. The NCSC on the risk of public large language models covers what happens to what you type into a public model, and it is worth reading before drawing that line.
If people are already using tools you have not approved, that is its own problem with its own shape, and shadow AI covers it.
Who checks it before it leaves?
An AI-assisted proposal, report or email still goes out under someone's name, and a client who spots an invented figure in it will not distinguish between the tool and the firm. So the useful rule is not "review AI output", because everyone agrees with that and nobody schedules it. It is naming which categories of work require a named person's sign-off before they go outside the organisation, and accepting that this costs time.
Where the work is internal and low-stakes, say so explicitly. A governance framework that demands the same review for a meeting summary and a regulatory filing gets ignored on both.
Which duties already apply?
More than people expect. The EU AI Act's transparency obligations attach to AI interactions and AI-generated content, and they have applied since 2 August 2026 (Article 50). They are a different set from the high-risk obligations that were deferred to 2 December 2027 and 2 August 2028 under Regulation (EU) 2026/1744, and the deferral did not touch them.
The AI literacy duty at Article 4 is live as well, and it reaches generative tools straightforwardly: if staff are using them, the duty is about whether those people have been given enough to use them sensibly. The AI literacy guide and the transparency guide both work through what that means in practice.
Whether any of it binds your organisation at all is a scope question first, and Article 2 settles that before anything else.