The most common question we get from regulated businesses about generative AI is not technical. It is political: Can we use it? Will the regulator allow it?
The short answer is yes — with conditions. The FCA, Gambling Commission, and GDS have none of them issued blanket prohibitions on generative AI. What they have done is make clear that existing obligations — around explainability, fairness, data protection, and consumer harm prevention — apply to AI-assisted decisions exactly as they apply to human-assisted decisions. The technology is new; the obligations are not.
The practical implication is that you cannot deploy generative AI in a regulated environment without first building the governance infrastructure that makes the AI's behaviour auditable, reversible, and explainable to a regulator who may not understand the technology but absolutely understands the obligation.
What Regulators Actually Require
Based on our engagements with FCA-regulated firms, GDS-assessed government services, and Gambling Commission licensees, the regulatory requirements for AI deployment cluster around four areas:
- Explainability: Any AI-assisted decision that affects a customer — a credit decision, a content moderation outcome, a responsible gambling intervention — must be explainable to that customer in plain English. You cannot satisfy this requirement with 'the model decided'.
- Data lineage: The regulator will want to know what data trained the model, whether that data was representative, and whether it contained protected characteristics in ways that could create discriminatory outputs.
- Human oversight: There must be a defined escalation path from AI-assisted decision to human review. The threshold for escalation must be documented, not improvised.
- Audit trail: AI-generated outputs that feed into regulated processes must be logged, versioned, and retrievable. 'We do not store model outputs' is not an acceptable answer.
The Governance Architecture
In practice, building this architecture means six things, in roughly this order:
1. AI Risk Classification
Not all AI use cases carry the same regulatory risk. A generative AI tool that drafts internal communications is categorically different from one that generates customer-facing decisions. Before building anything, classify your intended use cases by regulatory risk — and let that classification drive your governance requirements, not the other way around.
2. Data Governance Baseline
Most organisations want to deploy AI before their data governance is ready for it. Resist this. If you cannot answer the question 'what data trained this model and is that data accurate, representative, and legally obtained?' — you cannot deploy into a regulated environment. Full stop.
3. Model Evaluation Framework
Before any model goes into production, it needs to be evaluated against a defined set of criteria: accuracy benchmarks, bias tests, adversarial prompt testing, and output consistency checks. These evaluations need to be documented and retained, because a regulator may ask to see them.
4. Human-in-the-Loop Design
Define, before deployment, the cases where a human must review an AI output before it is acted upon. Then build that review into the product workflow — not as an afterthought, but as a first-class design element. A 'human override' button that nobody uses because the workflow makes it inconvenient is not a human-in-the-loop design.
5. Incident Response Protocol
AI models fail in ways that traditional software does not. They hallucinate. They behave inconsistently across superficially similar inputs. They degrade as the distribution of real-world inputs shifts from the training distribution. You need an incident response protocol that is specific to AI failure modes — not just a copy of your existing software incident process.
6. Regulatory Engagement
For any AI deployment in a material regulated process, engage with your regulator before you launch. The FCA's Innovation Pathways programme, the Gambling Commission's regulatory sandbox, and GDS's assessment process all provide mechanisms to test novel approaches before they become compliance problems.
The RAG Architecture That Actually Works
For most regulated enterprise use cases, Retrieval-Augmented Generation (RAG) is the right architecture. Here is why: a pure language model generates outputs from its training weights — which means it can hallucinate, cannot be audited against a specific knowledge base, and cannot be updated without retraining. A RAG architecture retrieves from a defined document corpus and generates responses grounded in retrieved evidence. The evidence is auditable. The retrieval can be logged. The outputs can be traced to source documents.
We have built RAG pipelines that reduced analyst reporting time by 60% for a regulated B2B platform — because the model was not generating free-form outputs, it was synthesising from a governed corpus of approved documents and labelling every output with the source passages it drew from. The compliance team could review any output, trace it to its sources, and verify it. That is what a regulated environment requires.
The technical choices that matter most: vector database selection (Pinecone and Weaviate are both solid; the choice depends on your infrastructure), chunking strategy (poor chunking destroys retrieval quality regardless of model quality), and re-ranking (a cheap retrieval model followed by a re-ranking step almost always outperforms a single expensive retrieval model).
The Honest Assessment
Generative AI delivers real value in regulated environments. We have seen it. But the organisations that realise that value are the ones that invest in governance infrastructure before they invest in model capability. The ones that skip the governance and race to deploy are the ones who call us six months later to help them unwind a deployment that has attracted regulatory attention or, worse, caused a customer harm incident.
Build the governance first. The AI will still work when you are done.