Does RAG require a vector database?

No. Retrieval can use keyword search, structured queries, vector similarity or combinations.

Can it use information from the web?

Yes, if the application provides web retrieval. Source quality and permissions still matter.

Does updating a file immediately update every answer?

Not necessarily. Index refresh, caches and document processing can introduce delays. Check how the particular system handles updates and removal.

AI Glossary / Retrieval and knowledge systems / Beginner

Retrieval-augmented generation

Definition

Retrieval-augmented generation, usually shortened to RAG, is a way of giving an AI model relevant information before it generates an answer. A retrieval step finds useful material, such as passages from approved documents, and the model uses that material as context. RAG can improve factual support, but it does not guarantee a correct answer.

What it means

Training does not automatically include your latest handbook or meeting records. RAG can find that information when someone asks a question.

The central idea is to combine finding with writing. Search identifies evidence; generation turns it into an explanation or draft. Either part can fail. The search may miss an exception, or the model may misread a passage that was retrieved correctly.

Adding a document to a searchable collection does not ordinarily retrain the language model. It changes the material available to retrieve. Some research systems train retrieval and generation components together, but retrieving passages for an answer is distinct from changing the model's learned parameters.

How it works

First, prepare the source material. Extract readable text, preserve useful headings and dates, and keep track of who may access each item. Dividing long documents into passages helps retrieve relevant sections without supplying entire files.

Next, use the question to search. The system might match keywords, compare numerical representations of meaning called embeddings, or combine approaches. Selected passages accompany the question and instructions.

Finally, the model writes an answer. Showing sources makes review easier. A citation still needs checking: the cited text must support the actual claim, apply to the right situation and be sufficiently current.

Access controls must apply before private material reaches an unauthorized user. Retrieved text is also untrusted input: instructions hidden inside a document should not be allowed to change permissions or authorize actions.

Three practical examples

Customer support: a retailer retrieves the policy for the customer's product category before drafting an explanation of the return process.

Personal organization: a household assistant searches appliance manuals to locate filter-cleaning instructions, showing the relevant model number and manual passage for confirmation.

Staff onboarding: an internal assistant finds the current procedure for requesting equipment, while respecting department-specific document permissions and directing unresolved exceptions to a person.

Think of it this way

Think of preparing an answer with a librarian and a writer. The librarian finds relevant pages; the writer explains them to the reader. Writing cannot repair missing evidence; finding evidence does not ensure accurate explanation.

Maintenance matters: outdated instructions with misleading titles can produce an apparently supported answer based on the wrong version.

Worked example

Checking retrieval separately from the answer

Suppose a fictional policy collection has eight passages that reviewers identify as relevant to a particular question. The system retrieves five passages. Three are relevant and two concern another policy.

Retrieval precision is the fraction of returned passages that are relevant: 3 ÷ 5 = 60%. Retrieval recall is the fraction of all relevant passages that were found: 3 ÷ 8 = 37.5%. These figures are invented, not a product benchmark.

Now imagine the omitted passages include an exception that changes the answer for temporary staff. The model may faithfully summarize the three relevant passages it received and still give an incomplete answer to that employee.

Adding more returned passages is not automatically a solution. Extra material can include duplicates, obsolete versions or distractions. Investigate the missed exception, improve retrieval or document preparation, and check the answer against the employee's circumstances.

Illustrative case study

Fictional case: Maple Equipment builds an assistant for warranty questions. Its collection contains current and older warranty guides with similar titles. The assistant sometimes cites the older version for a newly sold product.

The team records effective dates and product identifiers, removes obsolete documents from the current-policy search path, and tests questions involving repaired, refurbished and newly purchased products. It also displays the passage supporting each material condition.

Reviewers check both evidence and answer; a citation alone does not pass. This invented case illustrates source management, not measured improvement.

Common misconceptions and mistakes

RAG trains the model on my files. Indexing and retrieving files ordinarily supplies context at answer time; training is a different operation.

A source link proves the answer. The link may point to an irrelevant passage, an outdated rule or text that supports only part of the claim.

Vector search understands everything. Similarity can help find related language, but it can miss exact identifiers, dates, exceptions or important distinctions.

More documents always help. Additional conflicting or poorly prepared content can make retrieval and interpretation harder.

Business use

RAG is useful when answers depend on a changing or specialized collection of information. Common candidates include policies, product documentation and internal knowledge bases. Reliable, maintained sources are essential.

Before building an assistant, name the source owners, decide which versions are authoritative and define who may see each document. Include deletion and update procedures so the index does not silently preserve superseded information.

Evaluate retrieval and generation separately, then assess the complete experience. Test questions with no answer in the collection, conflicting sources and restricted documents. When evidence is insufficient, expose uncertainty or request review rather than inventing an answer.

Check your understanding

An assistant cites a handbook paragraph correctly, but misses a later paragraph containing an exception. Is the answer necessarily reliable because it has a citation?

Answer: No. Correct citation of one passage does not establish completeness. The missing exception may change the conclusion, so review must consider whether all necessary evidence was found and used.

Related terms

Grounding
Embedding
Hallucination
Access Control
Fine-tuning
Retrieval
search index
vector search
document chunking
citation

Learn next

Choose ten questions about a small, non-sensitive document collection. Mark the passages needed to answer each question before trying an assistant. Compare what it retrieves with those passages, then separately check whether its answers are supported and complete.

Sources

Lewis and colleagues, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks: https://arxiv.org/abs/2005.11401 . Original research abstract consulted for combining retrieval with generation; its training recipe is distinct from ordinary document indexing.

Microsoft, RAG and Generative AI: https://learn.microsoft.com/en-us/azure/search/retrieval-augmented-generation-overview . Primary guidance on retrieval, preparation and access controls.

OWASP, LLM Prompt Injection Prevention: https://cheatsheetseries.owasp.org/cheatsheets/LLM_Prompt_Injection_Prevention_Cheat_Sheet.html . Retrieved-content risks and permissions. All figures and cases are original illustrations.

Previous: Retrievability Next: Retrospective study