How RAG grounds an LLM with external knowledge
What is retrieval-augmented generation (RAG), and how does a typical RAG pipeline work?
what they're testing: The interviewer is probing whether you understand how retrieval and generation fit together to ground an LLM without retraining it.
RAG gives an LLM relevant outside material at the moment it answers. The application first prepares a knowledge base: documents are cleaned, split into chunks, and indexed. Embeddings and vector search are common, but retrieval can also use keyword, semantic, or hybrid search.
When a question arrives, the application searches that index for the most relevant chunks. It then puts those chunks beside the question in the model input. The LLM generates an answer from that augmented context, often with source metadata for citations. The model itself is not being retrained, so updated or private content can be made available by changing the knowledge base.
Retrieval quality is critical. If chunking loses context or search returns the wrong passages, the generator has poor evidence. RAG can reduce unsupported answers, but it cannot guarantee truth: the model may still ignore, misread, or invent beyond the retrieved material. A production system therefore evaluates retrieval and generation separately, applies access controls during retrieval, and checks the answer against the retrieved evidence.
Where people slip
the tempting wrong answer, and what's actually true
RAG trains the LLM on your documents so the model remembers them.
RAG retrieves relevant content at query time and places it in the model input without retraining the model.
A RAG system must use embeddings and a vector database.
Vector retrieval is common, but RAG can use keyword, semantic, vector, or hybrid search over an index or data store.
RAG eliminates hallucinations because every answer comes from retrieved documents.
Retrieved context reduces guessing, but irrelevant evidence or unsupported generation can still produce an inaccurate answer.
In RAG, the LLM searches the knowledge base and fetches the documents itself.
In a typical RAG pipeline, the application retrieves content first and then sends that content to the LLM in an augmented prompt.
If they push further
How is RAG different from fine-tuning?
RAG supplies knowledge at query time, which suits private or changing facts. Fine-tuning changes model behavior or task performance through training rather than acting as a live knowledge lookup.
What usually causes a RAG answer to fail?
Separate retrieval failures from generation failures. Check whether the right evidence was indexed and retrieved before judging whether the model followed it faithfully.
Does RAG require vector search?
No. Keyword, semantic, vector, and hybrid retrieval are all valid; the choice depends on the corpus, query patterns, filters, latency, and evaluation results.
Sources
- AWS Prescriptive Guidance: Understanding Retrieval Augmented Generation ↗docs.aws.amazon.com
- Microsoft Learn: Retrieval augmented generation and indexes ↗learn.microsoft.com
- Google Cloud: What is Retrieval-Augmented Generation? ↗cloud.google.com
- Microsoft Learn: Build advanced retrieval-augmented generation systems ↗learn.microsoft.com
Now answer it yourself.
Reading a strong answer is easy. Producing one under pressure is the skill the interview tests. Gapmap grades your answer against the same bar an interviewer would.
beta
The interviewer part is in the works.
The diagnostic, personal maps, and AI mock interviews are being finished right now. The notes stay free either way. Leave an email and you'll get the first-cohort invite, plus a month of Pro when it opens.