Retrieval-augmented generation (RAG) is the most practical way for a UK business to build an AI knowledge assistant that answers questions from its own documents — policies, contracts, product manuals, HR handbooks, project files and support tickets — rather than from whatever a general-purpose model happened to absorb during training. Instead of fine-tuning a model or hoping a chatbot “knows” your business, a RAG system searches your approved content at the moment a question is asked, hands the most relevant passages to a large language model, and asks it to answer using only that evidence, with citations back to the source.
This guide is a practical how-to for directors, operations leads and IT managers who want to move from “we should do something with AI and our documents” to a working, governed internal assistant. You will learn what RAG actually is, how it compares with fine-tuning and off-the-shelf tools such as Microsoft 365 Copilot, what a realistic build costs in pounds, the eight-stage implementation timeline we use, how to measure answer quality, and the UK GDPR, ICO and security considerations that decide whether the project survives contact with your data protection officer.
What is RAG? A plain-English definition for business owners
A large language model (LLM) such as GPT-4-class models, Claude or Gemini is very good at reading, summarising and writing, but it has two weaknesses that matter in a business setting. First, it knows nothing about your internal documents. Second, when it does not know something it may produce a fluent, confident answer that is simply wrong — usually called a “hallucination”. Retrieval-augmented generation addresses both problems by separating knowing from writing. Your documents are the source of knowledge; the model is the writer.
In practice a RAG pipeline has two halves. The ingestion half runs in the background: it collects documents from SharePoint, Google Drive, a CRM, a ticketing system or a file share, splits them into passages (“chunks”), converts each chunk into a numerical fingerprint called an embedding, and stores those fingerprints in a vector index alongside the original text and its permissions. The query half runs when a user asks a question: the question is embedded the same way, the index returns the closest-matching chunks, a re-ranker sorts them by relevance, and the model is instructed to answer from those chunks alone and to cite them.
The result is an assistant that can say “Our standard payment terms are 30 days from invoice, per section 4.2 of the 2025 Terms of Business” and link straight to the clause — or, just as importantly, say “I can’t find that in the documents you have access to” when the evidence is not there. That second behaviour is what makes RAG suitable for HR, finance, legal and customer-facing work where a wrong answer has real consequences.
RAG is not a product you buy; it is an architecture. Microsoft 365 Copilot, Google’s Gemini for Workspace, Glean, Notion AI and dozens of customer-service platforms all use a form of it under the hood. Building your own makes sense when your knowledge lives in systems those tools do not reach, when you need tighter control over sources and answers, or when you want to embed the assistant inside a customer portal, a line-of-business app or a field-engineer workflow.
Before anyone talks about models or vector databases, write down ten real questions staff asked last month and where the correct answer lives. If you cannot name the source document for at least seven of them, your first project is content clean-up, not AI.
RAG readiness scoring — where most UK businesses sit today
Most RAG projects that stall do so because of the documents, not the model. When we run readiness reviews for UK SMEs and mid-market firms, the same patterns show up: content is spread across four or five systems, there are three versions of the same policy, permissions were set years ago and never reviewed, and nobody owns the “source of truth”. The score cards below summarise how we grade the common readiness factors, from the ones that usually block a project to the ones that are normally in reasonable shape.
Use them as a quick self-assessment. Any “High risk” item is worth fixing before a pilot goes to more than a handful of users; “Medium” items can be fixed in parallel with the build; “Low” items rarely hold a project back.
The pattern is consistent: technology is rarely the constraint. Over-shared SharePoint sites are the single biggest risk, because a RAG assistant faithfully surfaces anything a user can technically open — including the salary spreadsheet someone saved to a team site in 2019. Fixing permissions is the same work you would need for a safe Microsoft 365 Copilot rollout, so it is never wasted effort.
RAG by the numbers — the figures that shape a business case
A business case for an AI knowledge assistant rests on a few practical numbers: how much time staff spend looking for information, how much an assistant costs to run per question, and how quickly a sensible pilot can be stood up. The figures below are indicative ranges drawn from our own SME and mid-market projects and from published vendor pricing in 2026; your numbers will differ, which is exactly why the pilot should measure them.
Two things stand out. Running costs per question are low — at a few pence per answer, even a busy 200-person firm asking several thousand questions a month spends tens or low hundreds of pounds on model calls. The real costs are in design, integration, content preparation and ongoing ownership, which is where the budget conversation should focus. And quality targets matter more than speed: an assistant that answers 95% of questions but is wrong on one in five will be abandoned within weeks, while one that answers 80% correctly and openly declines the rest builds trust.
RAG vs fine-tuning vs off-the-shelf tools — how they compare
The first architectural decision is whether you need RAG at all. There are three broad routes to an AI assistant that knows about your business. Off-the-shelf tools such as Microsoft 365 Copilot or Gemini for Workspace index the content already in their ecosystem and require little build work. Fine-tuning retrains a model on your examples so it adopts your style, terminology or task format. Custom RAG keeps the model general and feeds it your documents at question time.
Fine-tuning is frequently misunderstood. It is excellent for teaching a model a consistent output format, a tone of voice or a narrow classification task, but it is a poor way to teach facts: the knowledge becomes baked into the model weights, cannot be cited, goes stale the moment a policy changes, and cannot respect per-user permissions. For “answer questions from our documents” RAG is almost always the right foundation, sometimes with light fine-tuning layered on top later.
Fine-tuning a model
Retrain on your examples
Retrieval-augmented generation
Search your documents at question time
Where does Microsoft 365 Copilot fit? If most of your knowledge already sits in SharePoint, OneDrive, Teams and Outlook, and your users work inside Microsoft apps all day, Copilot is a sensible first step — and the measurement approach in our Copilot ROI guide applies equally to a custom build. Custom RAG earns its keep when the knowledge lives in a CRM, a ticketing system, an ERP, a document-management platform or a bespoke database; when you need the assistant in front of customers or partners; when you want strict control over which sources are used; or when licence costs for every seat do not add up for occasional users.
These options are not mutually exclusive. A common pattern is Copilot for everyday Office work plus a focused custom RAG assistant for one high-value domain, such as technical support, bid writing or compliance queries.
Where time goes — the use cases with the strongest RAG business case
Not every knowledge problem deserves an AI assistant. The strongest cases share three traits: questions are frequent, answers already exist in written form, and getting the answer wrong or slowly has a measurable cost. The chart below ranks the use cases we see most often in UK businesses by how well they fit those three traits — a higher score means a clearer, faster business case for a first RAG project.
Internal helpdesks sit at the top because the questions repeat endlessly (“How do I book annual leave?”, “What’s the VPN setting for home working?”), the answers sit in handbooks and runbooks, and each deflected ticket saves a few minutes of skilled time. Customer support follows closely, with the added benefit that a well-grounded assistant can draft replies for agents to approve rather than answering customers directly on day one.
Open-ended strategy research sits at the bottom not because AI cannot help, but because the answers are not in your documents — they require judgement, external data and synthesis. That is a different kind of project. Bid writing and contract lookup are high-value but carry more risk, so they suit a second phase once you have proved retrieval quality and governance on a lower-stakes domain.
Why answer quality fails — the anatomy of a bad RAG response
When a RAG assistant gives a poor answer, the instinct is to blame the model and switch to a larger one. In our experience that is rarely the fix. Tracing failed answers back through the pipeline usually shows that the problem started much earlier: the right document was never indexed, it was chunked in a way that split the key sentence from its heading, the search returned a near-miss, or the prompt allowed the model to fill gaps from general knowledge.
That figure should change how you allocate effort. Roughly seven in ten bad answers we trace are fixable by improving the documents, the way they are split, the metadata attached to them or the search strategy. Only a minority are genuinely model problems — misreading a table, failing to combine two passages, or ignoring an instruction. This is good news: content and retrieval fixes are cheap, testable and do not depend on the next model release.
The practical lesson is to build an evaluation set early (more on that below) and to log, for every answer, which chunks were retrieved. When a user flags an answer as wrong, you can then see immediately whether the right evidence was in front of the model. If it was not, fix retrieval. If it was, fix the prompt or the model choice.
How to build a RAG knowledge assistant — the step-by-step timeline
The timeline below is the eight-stage approach we use for a first RAG assistant in a UK SME or mid-market organisation, covering one domain, three to five source systems and a pilot group of 20 to 50 users. Each stage has a clear output so the sponsor can see progress and stop or change direction at any gate. Larger estates, customer-facing assistants or regulated domains add time, mainly in stages two, six and seven.
Notice that the first two weeks involve almost no AI engineering. That is deliberate. Teams that skip straight to a demo usually build something impressive on ten hand-picked documents, then discover in week six that the real estate contains 40,000 files with tangled permissions and six versions of the expenses policy. Front-loading the question bank and content audit is the single most effective way to keep a RAG project on schedule.
Keep the question bank in a simple spreadsheet with three columns: question, expected answer, source document. It doubles as the test suite every time you change a model, a prompt or a chunking rule.
The technical choices explained without the jargon
You do not need to become a machine-learning engineer to commission a RAG assistant, but understanding the main design choices helps you ask the right questions of any supplier, including us. Each of the decisions below has a sensible default for most UK businesses.
Chunking: how documents are split
Models can only read a limited amount of text per question, so documents are cut into passages. Splitting blindly every 500 words often separates a rule from the heading that gives it meaning. Better approaches split on headings and sections, keep tables intact, and include a small overlap between chunks. Each chunk should carry its document title and section heading so the model knows where it came from.
Embeddings and the vector index
An embedding model turns each chunk into a list of numbers that captures its meaning, so “annual leave” and “holiday entitlement” land close together. Those vectors live in an index — Azure AI Search, PostgreSQL with pgvector, Cloudflare Vectorize, Pinecone and others all work. For most SMEs the choice of index matters far less than the quality of the content going into it; pick one that sits in your existing cloud, supports metadata filtering and offers UK or EU hosting.
Hybrid search and re-ranking
Pure vector search is good at meaning but can miss exact terms such as product codes, policy numbers or surnames. Hybrid retrieval combines vector similarity with traditional keyword search, then a re-ranker reads the top 20–50 candidates and orders them by genuine relevance. This combination is the single biggest quality improvement we see over naive RAG demos.
Permission-aware retrieval
Every chunk should carry the access groups of its source document, and every search should filter by the asking user’s groups before anything reaches the model. Never rely on the prompt to tell the model to “ignore documents the user cannot see” — if the text is in the prompt, it can leak into the answer.
Grounded prompting and citations
The answer prompt instructs the model to use only the supplied passages, to cite each claim, and to say clearly when the evidence is missing or contradictory. Citations should link to the original document so users can check, and the interface should make those links prominent rather than tucked away.
Model choice
Mid-sized models from the major providers are usually sufficient for grounded question answering, with a larger model reserved for complex multi-document reasoning. Prefer providers that offer UK or EU processing, a contractual commitment not to train on your data, and a data processing agreement your DPO is comfortable with. Design the system so the model can be swapped without rebuilding the pipeline.
RAG readiness gauge — the benchmark score to aim for before launch
To decide whether an assistant is ready to move beyond the pilot group, we combine five measures into a single readiness score out of 100: retrieval hit rate on the question bank (the right source appears in the top five results), grounded-answer rate (answers fully supported by cited passages), correct refusal rate (the assistant declines when the answer is not in the documents), permission-leak tests passed, and pilot user satisfaction. Each is weighted, with retrieval and grounding counting most.
A score of 76 is not a pass mark for perfection; it is the point at which, in our experience, users trust the assistant enough to keep using it and the remaining failures are visible and explainable. Below roughly 60, users start reporting the assistant as “unreliable” and usage falls away. Above 85, the remaining improvements are usually about content coverage — adding the next document set — rather than engineering.
Whatever threshold you choose, agree it with the sponsor before the pilot starts. It turns a subjective “is it good enough?” debate into a measured decision, and it stops a project drifting into months of polishing without a clear finish line.
RAG cost breakdown — what a UK business should budget
Costs vary with the number of source systems, the volume and messiness of the content, the security and compliance requirements, and whether the assistant faces staff or customers. The table below gives indicative 2026 ranges for UK projects delivered by a specialist partner, excluding VAT and excluding any licences you already hold.
| Scope | Typical profile | Indicative build cost | Indicative monthly running cost |
|---|---|---|---|
| Proof of value | One domain, one source system, 10–20 testers, 4–5 weeks | £8,000–£15,000 | £50–£250 |
| Departmental assistant | One domain, 3–5 sources, permissions, Teams app, 50–150 users | £18,000–£35,000 | £250–£900 |
| Customer-facing assistant | Public knowledge base, portal integration, guardrails, escalation to agents | £25,000–£45,000 | £400–£1,500 |
| Multi-domain platform | Several domains, 8+ sources, analytics, admin console, SSO | £45,000–£120,000 | £1,000–£4,000 |
Running costs include model usage, embedding refreshes, the search index, hosting and monitoring. They scale mostly with the number of questions and the volume of content re-indexed, and at SME volumes they are modest. The line that is easy to forget is ownership: someone needs to review flagged answers, keep content current and decide when to add new sources. Budget a few hours a week of a content owner’s time, or include it in a managed support agreement.
Compare these figures with seat-based alternatives. Microsoft 365 Copilot is licensed per user per month; for a 150-person firm where only 40 people would use an assistant regularly, a focused custom RAG tool may cost less over three years, while for a firm whose knowledge is entirely in Microsoft 365 and whose staff use Office all day, Copilot is likely to win. Our AI software development cost guide covers the wider pricing picture.
Be cautious of quotes that cover only the “chatbot” and leave content preparation, permissions work and evaluation to you. Those are the stages that decide whether the assistant is trusted.
RAG benchmarks and KPIs to track from day one
An assistant that is not measured slowly degrades: documents change, new questions appear, and nobody notices until users stop trusting it. The benchmarks below are the targets we typically set for a well-run internal assistant three months after launch. Treat them as starting points to agree with your sponsor rather than universal standards.
Target operating benchmarks for an internal RAG assistant
Two of these deserve special attention. Permission-leak tests must sit at 100%: a single case of a junior employee retrieving board papers or payroll data is a reportable incident and a trust-destroying event. Automate these tests with dedicated test accounts in each permission group and run them after every change. Correct refusals are the second: an assistant that confidently answers questions it should decline will eventually say something wrong about a policy, a price or an entitlement, and someone will act on it.
Business measures sit on top of these quality measures: tickets deflected from the IT or HR helpdesk, average handling time in customer support, time-to-first-draft for bids, onboarding time for new starters. Agree one or two of these with the sponsor at the start and capture a baseline before launch, otherwise you will have no way to show value later.
Common RAG mistakes UK businesses make
Most of the problems we are asked to fix in existing RAG projects fall into a short list. None of them are exotic; all are avoidable with a little planning.
- Indexing everything on day one. Pointing the assistant at every SharePoint site and file share produces a noisy index full of drafts, duplicates and out-of-date material. Start with one curated domain and grow.
- Ignoring permissions until late. Retro-fitting access control into a working pipeline is painful and risky. Design permission-aware retrieval from the first ingestion run.
- No evaluation set. Without a fixed list of real questions and expected answers, every change is judged on gut feel and the loudest anecdote. Build the question bank in week one.
- Blaming the model first. Upgrading to a larger model is expensive and usually does not fix poor chunking or missing documents. Trace failures through retrieval before changing models.
- Hiding the sources. If users cannot see and click the documents behind an answer, they cannot verify it, and trust erodes after the first mistake. Make citations prominent.
- Skipping the DPIA. Processing HR records, customer correspondence or health information through an AI service without a data protection impact assessment exposes the business to ICO scrutiny and internal objections late in the project.
- Forgetting prompt injection. Documents and user inputs can contain instructions that try to override the system prompt. Treat retrieved content as untrusted data and test for this explicitly, just as you would for web application vulnerabilities.
- No owner after launch. Without a named person reviewing flagged answers and maintaining content, quality drifts downwards within a few months.
A RAG assistant inherits every permission mistake in your file estate. If “Everyone except external users” can open a folder, the assistant will happily summarise it for anyone who asks. Audit sharing on in-scope sources before the first ingestion run, not after the first complaint.
UK GDPR, the ICO and security — doing RAG responsibly
RAG is often presented as the “safe” way to use AI with company data, and architecturally it is safer than pasting documents into public chatbots. But it still involves processing information — frequently including personal data — through third-party services, and it creates new records in the form of prompts, retrieved passages and answers. UK GDPR applies in full.
Lawful basis and purpose
Identify the lawful basis for processing any personal data in the indexed content, and check that answering staff questions is compatible with the purpose for which it was originally collected. For most internal knowledge bases this is legitimate interests, supported by a documented balancing test.
Data protection impact assessment
The ICO’s guidance on AI and data protection expects a DPIA where processing is likely to result in high risk, and new technologies such as AI are a recognised trigger. Even where it is not strictly required, a short DPIA is the clearest way to record your decisions on data residency, retention, access control and supplier terms — and to bring your DPO on side early.
Supplier terms and data residency
Use enterprise or API agreements that commit the provider not to train on your data, specify retention of prompts and outputs, and offer UK or EU processing where you need it. Record international transfers and the safeguards in place, such as the UK International Data Transfer Agreement or the UK Extension to the EU–US Data Privacy Framework where applicable.
Security controls
Treat the assistant as a business application: single sign-on with MFA, conditional access, logging, least-privilege service accounts for connectors, secrets in a vault, and regular testing. The controls required for Cyber Essentials — supported software, secure configuration, access control — apply to the servers and services that run it. The NCSC’s guidelines for secure AI system development are a useful checklist for the design review.
Subject access and retention
Prompts and answers can contain personal data and may fall within scope of a subject access request. Decide how long logs are kept, who can see them and how they would be searched if a request arrived. Shorter retention with aggregated analytics is usually the sensible default.
RAG implementation checklist — the 12-point essentials
Use this checklist as a gate before you widen a pilot or put an assistant in front of customers. Every item should have a named owner and a yes/no answer.
- One domain chosen with a director-level sponsor and a written success measure.
- Question bank of 50–150 real questions with expected answers and source documents.
- Authoritative content identified, duplicates archived and each content set given an owner.
- Permissions reviewed on every in-scope source, with over-shared folders fixed.
- DPIA completed and signed off, with lawful basis and retention recorded.
- Supplier agreements in place covering no-training commitments, residency and processing terms.
- Heading-aware chunking with document title and section metadata on every chunk.
- Hybrid retrieval and re-ranking configured and tested against the question bank.
- Permission filtering at retrieval, verified by automated leak tests for each access group.
- Grounding and citation rules in the prompt, with an explicit “I don’t know” behaviour.
- Prompt-injection and abuse testing completed and findings resolved.
- Feedback, logging and ownership in place, with a weekly review of flagged answers.
If a virtual CIO or IT steering group oversees your technology roadmap, this checklist belongs on its agenda. It fits neatly into the governance rhythm described in our virtual CIO first 90 days guide.
Mini case study — an internal helpdesk assistant for a Midlands engineering firm
A 140-person precision engineering business in the West Midlands came to us with a familiar problem. Its small IT and HR teams fielded hundreds of repetitive questions a month by email and Teams — holiday rules, expense limits, how to request access to the ERP, how to connect to the shop-floor Wi-Fi. The answers existed, but in an employee handbook last restructured in 2021, a set of IT runbooks in a file share, and a SharePoint site with three overlapping “Policies” libraries.
The first two weeks were spent almost entirely on content. With the HR manager and IT lead we collected 120 real questions from the previous quarter’s tickets, mapped each to a source, and found that 23 had no current written answer at all. Those gaps were filled before any AI work started. Duplicate policy libraries were merged, two folders shared with the whole company were locked down, and a DPIA was completed with the firm’s external DPO.
The assistant itself was delivered as a Teams app, indexing the handbook, IT runbooks and policy library with permission-aware hybrid search. Every answer carried links to its sources and a thumbs-up/down button. On the question bank, the first build scored a retrieval hit rate in the low 80s; after re-chunking the handbook by section and adding section titles to each chunk it rose above 90%, without changing the model. The pilot ran with 35 staff for four weeks before rolling out company-wide, and the HR manager now spends an hour a week reviewing flagged answers and updating content.
“The surprise for us was that the AI part was the easy bit. Fixing the handbook and the SharePoint permissions took longer than building the assistant — but that work would have been worth doing anyway, and now people actually find the answers.” — Operations Director, West Midlands engineering firm (anonymised)
The lesson generalises: the measurable value came from making knowledge findable and trustworthy. The assistant was the interface that made that work visible to everyone.
RAG at a glance — the key facts in one table
| Topic | Key fact or recommendation |
|---|---|
| What RAG does | Searches your approved documents at question time and asks a model to answer only from them, with citations |
| Best first use cases | Internal IT/HR helpdesk, customer support knowledge, field engineer manuals |
| RAG vs fine-tuning | RAG for facts and current documents; fine-tuning for style, format or narrow tasks |
| RAG vs Microsoft 365 Copilot | Copilot suits knowledge already in Microsoft 365; custom RAG suits other systems, customers and focused domains |
| Typical pilot timeline | 6–10 weeks for one domain, 3–5 sources and 20–50 users |
| Indicative build cost | £8,000–£15,000 proof of value; £18,000–£35,000 departmental assistant |
| Indicative running cost | A few pence per question plus index, hosting and monitoring |
| Biggest quality lever | Content clean-up, heading-aware chunking and hybrid retrieval with re-ranking |
| Biggest security risk | Over-shared files surfaced by the assistant; enforce permissions at retrieval |
| Compliance essentials | DPIA, lawful basis, supplier no-training terms, UK/EU residency, log retention |
| Readiness benchmark | Agree a measured threshold (we use 76/100) before widening the pilot |
| Ongoing ownership | A named content owner reviewing flagged answers weekly |
How Cloudswitched delivers RAG knowledge assistants
Cloudswitched designs and builds AI knowledge assistants for UK businesses, starting with the question bank and content audit rather than the model. We handle connectors to Microsoft 365, Google Workspace, CRMs, ticketing systems and line-of-business databases, permission-aware hybrid retrieval, grounded answer design, evaluation, security testing and the DPIA support your DPO will ask for — and we can run the assistant afterwards as a managed service, alongside your wider IT support.
Get expert help building your RAG assistant
Talk to our AI software team about scoping a governed proof of value on your own documents.
Talk to us about AI Software DevelopmentFrequently Asked Questions
What is retrieval-augmented generation in simple terms?
Retrieval-augmented generation is a way of building an AI assistant that looks up the answer in your own documents before it writes a reply. When someone asks a question, the system searches an index of your approved content, picks the most relevant passages and gives them to a language model with instructions to answer only from that evidence and cite it. The model provides the writing ability; your documents provide the facts. This keeps answers current, lets users check sources and reduces the risk of confident but wrong answers.
Is RAG better than fine-tuning for a business knowledge base?
For answering questions from company documents, RAG is almost always the better starting point. Fine-tuning bakes information into the model itself, which means it cannot cite sources, goes out of date whenever a policy changes and cannot respect per-user permissions. RAG keeps the knowledge in your documents, so updating a policy updates the answers within minutes. Fine-tuning remains useful for teaching a consistent style, output format or narrow classification task, and it can be layered on top of RAG later if needed.
Do we need a custom RAG system if we already have Microsoft 365 Copilot?
Not necessarily. If your knowledge lives mainly in SharePoint, OneDrive, Outlook and Teams and your staff work in Office all day, Copilot may cover most needs once permissions are tidied. A custom RAG assistant makes sense when key knowledge sits in other systems such as a CRM, ticketing platform or ERP, when you want an assistant in front of customers or partners, when you need strict control over which sources are used, or when per-seat licensing does not suit occasional users.
How much does a RAG knowledge assistant cost in the UK?
As an indication, a focused proof of value typically costs £8,000 to £15,000, a departmental assistant with several sources, permission-aware search and a Teams interface £18,000 to £35,000, and a customer-facing assistant £25,000 to £45,000. Running costs at SME volumes are usually modest — often a few hundred pounds a month for model usage, search and hosting. The cost most often underestimated is content preparation and ongoing ownership, so make sure any quote is clear about who does that work.
How long does it take to build a RAG assistant?
A governed pilot covering one domain, three to five source systems and 20 to 50 users typically takes six to ten weeks. The first two weeks are mostly scoping, building a question bank, cleaning content and completing the DPIA. Ingestion, retrieval and answer design follow, then evaluation, security testing and a live pilot. Projects with very messy content, many systems or customer-facing requirements take longer, mainly because of content and governance work rather than AI engineering.
Can a RAG assistant leak confidential information?
It can if it is badly designed. The assistant can surface anything that is indexed and not filtered, so the essential controls are reviewing permissions on source systems, storing access groups with every chunk and filtering search results by the user’s permissions before anything reaches the model. Relying on the prompt to hide content is not sufficient. Automated leak tests using accounts in each access group should pass every time the system changes, alongside testing for prompt injection.
Do we need a DPIA for an AI knowledge assistant?
If the indexed content includes personal data — HR records, customer correspondence, case notes — a data protection impact assessment is strongly advisable and often required, because the ICO treats innovative technologies such as AI as a factor indicating high-risk processing. Even when it is not mandatory, a DPIA is the clearest place to record decisions on lawful basis, data residency, supplier terms, retention of prompts and logs, and access control, and it helps bring your DPO into the project early.
How do we stop the assistant making things up?
Three measures do most of the work. First, good retrieval — clean content, heading-aware chunking, hybrid search and re-ranking — so the right evidence reaches the model. Second, a strict prompt that requires the model to answer only from the supplied passages, cite them and say clearly when the answer is not there. Third, an evaluation set of real questions run after every change, with a target for grounded answers and correct refusals. No system is perfect, which is why visible citations and a feedback button matter.
Which documents should we index first?
Start with one domain where questions are frequent, answers are already written down and the content has a clear owner — an employee handbook and IT runbooks for an internal helpdesk, or a product knowledge base for customer support. Index the authoritative versions only, archive duplicates, and make sure scanned files have searchable text. Resist the temptation to connect every SharePoint site at once; a smaller, curated index produces better answers and is easier to govern.
Where should our data and the AI model be hosted?
Choose providers that offer UK or EU processing where your data protection position requires it, contractual commitments not to train on your data, and clear retention terms for prompts and outputs. Hosting the index and application in the cloud you already use — Azure, AWS, Google Cloud or Cloudflare — usually simplifies identity, networking and monitoring. Record any international transfers and their safeguards in your DPIA and records of processing.
What skills do we need internally to run a RAG assistant?
You do not need data scientists. You do need a sponsor who owns the success measure, content owners who keep documents current and review flagged answers, and IT staff or a partner to manage identity, permissions, monitoring and updates. Budget a few hours a week of content owner time after launch. Many SMEs choose to have a partner run the technical platform as a managed service while their own teams own the content.
Related reading
If you are planning an AI knowledge assistant, these recent Cloudswitched guides cover the adjacent decisions on adoption, measurement, security and governance.
Turn your documents into a trusted AI assistant
Cloudswitched builds governed, permission-aware RAG assistants for UK businesses — from content audit and DPIA through to a measured pilot and managed support.
Explore our AI Software Development service
Cloudswitched
London-based managed IT services provider offering support, cloud solutions and cybersecurity for SMEs.
