Retrieval-Augmented Generation (RAG)

ELI5 (Explain Like I'm 5)

Think of a Customer Support Agent handling a ticket.

• Standard LLM: The agent tries to answer the customer's technical question entirely from memory. They might sound confident, but they could be referencing an old manual.
• RAG: Before answering, they type the customer's error code into your internal Knowledge Base, pull up the article written recently, and read the solution to the customer.

RAG is meant for proprietary data like your PDFs, SQL databases, and internal wikis.

What is RAG?

Retrieval-Augmented Generation is the concept used to bridge the gap between an LLM's outdated training data and your private data.

It solves the knowledge cutoff problem. Instead of retraining a model every time you have new data, RAG separates knowledge from reasoning.

Why not retraining model?

Well, two main reasons:

1. It is time-consuming.
2. It takes massive computing, hence, expensive.

RAG is a 3-step process:

1. Retrieval: The user asks a question. The system searches your database (usually a Vector Database) for the most relevant documents or data chunks.
2. Augmentation: Those relevant chunks are pasted into the prompt as context.
3. Generation: The LLM reads the context and generates an answer based only on the information provided, citing its sources.

Modern RAG has evolved into GraphRAG (using Knowledge Graphs) and Agentic RAG (where an agent actively searches and filters data), improving accuracy significantly over the simple vector searches.

RAG Builder's Stack

To build a RAG pipeline, you can leverage the following tools.

Pain point by community

Retrieval-augmented generation (rag) is a hack; it works today, but it doesn't make sense long-term to encode data into one representation, retrieve it to its original representation, and put it in the context window, which then encodes it again into the model's internal rep.

— Anton