HomeGlossaryRetrieval-Augmented Generation (RAG)
Glossary

What Is Retrieval-Augmented Generation (RAG)?

Retrieval-augmented generation (RAG) has an AI model look up relevant passages in your own documents at question time and answer from them, with citations.

Definition

What is retrieval-augmented generation?

Retrieval-augmented generation (RAG) is a way of building AI assistants that look up relevant passages in a company's own documents or data when a question is asked and give them to the language model to answer from. The model's knowledge is extended without retraining, and answers can cite their sources.

Full guide: RAG vs fine-tuning: building AI on your company data Service: Custom AI Applications

Retrieval-augmented generation (RAG) pairs a language model with a search step. NIST's glossary defines it as a type of generative AI system in which a model is paired with a separate information retrieval system, or knowledge base: based on a user query, the system identifies relevant information within the knowledge base and provides it to the model in context for the model to use in formulating its response. The term comes from a 2020 research paper by Patrick Lewis and colleagues.

The process has two stages. Before any question is asked, the company's content, such as policies, contracts, manuals or tickets, is split into passages and indexed, commonly as embeddings in a vector index alongside keyword search. When a user asks something, the system retrieves the passages most relevant to the question, adds them to the prompt and instructs the model to answer from that material. A good build shows the sources beside the answer and tells the model to say so when the documents do not contain the answer.

RAG is a common first choice for putting company knowledge behind an AI assistant because it needs no model training and the content can be updated by re-indexing. It also carries its own risks. Retrieval must respect permissions, or the assistant becomes a way to read files a user could never open. Old or duplicate documents produce confident answers from the wrong version. And retrieved text is untrusted input: a document containing hidden instructions can steer the model, which is indirect prompt injection. The OWASP Top 10 for LLM Applications has a category for vector and embedding weaknesses. A business may already be using RAG without calling it that, since Microsoft 365 Copilot grounds its responses in the organizational data a user can already access.

Reviewed by Jamie Baum, Head Implementation Manager, AI Practice. Our editorial standards.

Why it matters for a small business

A general AI model knows nothing about your clients, prices or procedures, and when asked it may invent an answer. RAG is how you get answers from your own material with a source you can check, without sending your data off to train a model. For an owner the decision is less about the technology than about the content behind it: which document libraries the assistant may read, whether permissions on them are clean and who keeps them current. An assistant built on a messy file share gives messy answers faster.

How NetSys builds on company data

When NetSys builds an assistant or reporting tool on a client's data, answers are grounded in retrieved records and cite their source, so a user can check them. The model is instructed to say when the data holds no answer instead of guessing, and numbers in reports come from calculations run on the data, never from the model's memory.

Connections run through Microsoft Graph, vendor APIs or a read replica, read-only unless a write is named, under least-privilege credentials, and prompts are tested against injected instructions before rollout. Discovery ends in a one-page specification of the questions the tool must answer and the sources it may read. See our custom AI applications service.

Common Questions

Retrieval-Augmented Generation (RAG): FAQs

What is RAG in simple terms?

RAG lets an AI assistant look things up before it answers. Instead of relying only on what the model learned in training, the system searches your documents for passages relevant to the question and hands them to the model, which writes its answer from them and can show where each point came from.

What is the difference between RAG and fine-tuning?

RAG gives a model your information at the moment of the question by retrieving it; the model itself is unchanged. Fine-tuning retrains the model on examples so that its behavior or style changes. RAG suits facts that change and need citations. Fine-tuning suits a consistent format or tone.

Does RAG stop AI hallucinations?

It reduces them; it does not end them. A model answering from retrieved passages is less likely to invent facts, but it can still misread a passage, combine sources wrongly or answer from an outdated document. Showing citations, testing on real questions and instructing the model to decline when the sources are silent all help.

Is company data safe in a RAG system?

It depends on the build. Check that retrieval respects each user's existing permissions, that the model provider does not train on your prompts under the plan you use, that the index is stored and access-controlled like the documents it came from, and that documents entering the index are treated as untrusted input.

Custom AI Applications

Discuss custom ai applications for your business.

Tell us about your current systems, the result you need and your timeline. We will discuss the work, responsibilities and pricing before you decide on an engagement.

Explore Custom AI Applications 845-203-3914