What are the main stages of a RAG pipeline?

A RAG pipeline has four main stages: ingest (bringing raw documents into the system), embed (splitting documents into chunks and converting them into numeric vectors), retrieve (finding the chunks most relevant to a user’s query), and generate (giving those retrieved chunks to a language model so it can produce a grounded answer). The first two stages happen once, ahead of time; retrieval and generation happen fresh for every query.

AI & ML — INTRODUCTION

The RAG Pipeline: Ingest, Embed, Retrieve, Generate

Every piece of RAG covered in this series — chunking, embeddings, vector search — is one stage in a single connected pipeline. Here’s how a raw document actually becomes a grounded answer, start to finish.

The Quick Answer

A RAG pipeline has four stages: ingest (bring in raw documents), embed (split them into chunks and convert those into searchable vectors), retrieve (find the chunks most relevant to a question), and generate (hand those chunks to a model so it can answer using real text). The first two stages — ingest and embed — happen once, ahead of time, for a whole document collection. Retrieve and generate happen fresh, every single time someone asks a question.

What the Experts Say

A widely cited academic survey of RAG systems frames the whole approach around exactly this kind of structure, describing what it calls the “tripartite foundation of RAG frameworks” — retrieval, generation, and the augmentation step that connects them. Splitting ingestion and embedding out as their own stages (as this article does) simply makes explicit the preparation work that has to happen before retrieval is even possible. See Gao et al., “Retrieval-Augmented Generation for Large Language Models: A Survey,” arXiv:2312.10997.

The Four Stages, and Where Each One Is Covered

1
Ingest. Raw documents — PDFs, Markdown files, web pages — enter the system and get split into smaller, manageable pieces. See Document Chunking.
2
Embed. Each chunk gets converted into a numeric vector that captures its meaning, and stored where it can be searched later. See Embeddings.
3
Retrieve. A user’s question gets embedded the same way, then matched against every stored chunk to find the most relevant ones. See Vector Database.
4
Generate. The retrieved chunks are added to the model’s context, and it generates an answer grounded in that real, specific text. See RAG.

Follow One Document Through the Whole Pipeline

Step through all four stages below and watch the same piece of information — the XR-400’s battery life — transform at each one, from raw document to grounded answer.

Fun Fact

Ingest and embed only need to happen once per document — but retrieve and generate happen again for every single question, which is exactly why production systems separate these into two different processes: a slower, offline indexing job, and a fast, real-time query path.

Test Yourself

What are the four main stages of a RAG pipeline, in order?

Which stages happen once, ahead of time, rather than fresh for every user query?

Ready to see indexing-time vs query-time as two distinct data flows, and where the latency in each stage actually comes from? Continue to the Intermediate level →