BUILDER’S GUIDE  ·  RAG ARCHITECTURES

From Naive to Agentic: A Practical Guide to 14 RAG Architectures

Most teams do not fail at RAG because the LLM is weak. They fail because they picked the wrong retrieval pattern for the job. This guide maps fourteen architectures you will actually see in production — what each one does, when it earns its complexity, and when it is wasted spend.

Written as an original field guide for engineers, educators, and product teams building grounded AI systems. Not a rewrite of any vendor article — a builder’s map.

Why one RAG is never enough

Classic RAG is a three-step loop: embed the question, pull nearby chunks from a vector store, stuff them into a prompt, generate. That loop is enough when the question is short, the corpus is clean, and a wrong answer is cheap.

Real products are messier. Users ask follow-ups. Documents contradict each other. Answers live in a chart, not a paragraph. A contract clause only makes sense if you also know the related party. Those situations need extra loops: rewrite the question, score the evidence, walk a graph, call a tool, or generate a hypothetical document first.

Think of the fourteen types below as knobs, not a ladder you must climb. Start simple. Add a knob only when a failure mode shows up in evaluation.

The 14 architectures at a glance

1. Naive RAG

The shortest path from a question to an answer. Convert the query into search terms (sometimes not even embeddings), take the first matching documents, and hand them to the model with almost no filtering.

Use it when the knowledge base is small and questions are predictable — a 40-page policy handbook, an internal FAQ. Skip it the moment users start asking multi-part questions; naive retrieval will happily inject the wrong paragraph and the model will sound confident about it.

Keep if: you need a baseline in a day.

Drop if: irrelevant chunks keep leaking into answers.

2. Simple RAG (the original loop)

This is the version most tutorials teach. Embed the user question, run similarity search, take top-k chunks, generate once. No memory, no second look, no rewrite.

It is still the right default for product documentation, employee handbooks, and first-pass customer support. Fast to ship, cheap to run, easy to debug. The failure mode is structural: one retrieval pass cannot stitch an answer that lives across three documents.

Keep if: latency and cost matter more than multi-hop reasoning.

Drop if: your eval set is full of questions that need two or more sources.

3. Simple RAG with memory

The same loop, plus a conversation store. Past turns (and sometimes past retrieved chunks) are folded into the next search so “what about its population?” still means Paris.

This is the difference between a one-shot Q&A widget and a usable assistant. The cost is real: more tokens, more chance of dragging stale context forward, and a privacy surface you now have to govern.

Keep if: users talk in threads, not isolated tickets.

Drop if: sessions are short and each question is self-contained.

4. HyDE — Hypothetical Document Embeddings

Instead of embedding the raw question, the model first writes a short imaginary answer. That imagined paragraph is embedded and used as the search key. You are matching “what a good document would look like,” not the user’s wording.

HyDE shines when the corpus uses specialist language the user does not. A clinician types symptoms; the hypothetical note already sounds like a case report, so retrieval lands in the right neighborhood. The risk is obvious: a bad guess steers the search into a different disease.

Keep if: keyword and raw-query embeddings consistently miss.

Drop if: your queries already look like the documents.

5. Self-RAG

The system treats retrieval as something it can refuse or redo. After a first pass it scores whether the evidence actually supports an answer. If the question is underspecified, it rewrites the query and searches again. Some implementations even emit special tokens that mean “retrieve,” “enough,” or “I should not answer.”

Useful whenever users type half a thought. Expensive, because you pay for critique passes. Watch for over-refusal — Self-RAG can become a system that politely declines everything it is not 90% sure about.

6. Corrective RAG (CRAG)

Generate as usual, then audit the sources. Weak or off-topic chunks get dropped. If the remaining evidence is thin, the system runs a fresh retrieval — sometimes against the open web or a fallback index — and only then finalizes the answer.

This is the pattern you want when a wrong citation is worse than a slow answer: policy analysis, academic drafts, regulated industries. Guard against infinite “I’m still not happy” loops with a hard retry budget.

7. Speculative RAG

While the current answer is being written, the system guesses the next question and pre-fetches documents for it. When the user actually asks that follow-up, retrieval is already warm.

Great for live chat and guided troubleshooting. Wasteful if your guess rate is poor — you will burn embeddings on paths nobody takes. Measure hit-rate of pre-fetches before you keep this in production.

8. Adaptive RAG

A router sits in front of retrieval. Simple factoid? Cheap keyword or single-vector search. Multi-hop analysis? Graph or agentic path. Broad exploratory question? Wider recall and a heavier reranker.

Adaptive RAG is how mixed workloads stay affordable. The hidden work is labeling query types well enough that the router is right more often than a single default strategy. Early on, the router will be wrong; log those misses and treat them as training data.

9. Branched RAG

Ambiguous questions are split into parallel interpretations. Each branch retrieves and drafts. A judge (or a scoring function) picks or merges the strongest path.

This is how you stop missing a whole side of an open-ended market question. It is also how you flood a user with three essays if you do not collapse the branches before display. Always budget a synthesis step.

10. Graph RAG

Documents are not just chunks. They are nodes and edges: companies own products, clauses amend clauses, genes interact with pathways. Retrieval walks those links, so an answer can include a document that never shared keywords with the question but sits one hop away in the graph.

Build this when relationships are the product — investigations, competitive intelligence, biomedical literature. Do not build it as decoration. A sloppy graph is slower than simple RAG and no more accurate.

11. Multimodal RAG

The index is no longer text-only. Pages, figures, screenshots, tables, and sometimes audio are embedded (or captioned, then embedded). A question about “the red line on last quarter’s chart” can retrieve the image and the surrounding paragraph together.

This is the only honest architecture for manuals, slide decks, and scanned reports. Storage and embedding cost jump. Quality tracks how well you parse layout — a bad OCR pipeline will poison every downstream answer.

12. Modular RAG

Retrieval, reranking, compression, memory, and generation are separate services with stable interfaces. You can swap a BM25+vector hybrid retriever for a late-interaction model without rewriting the generator. You can add a reranker on one tenant and leave it off for another.

This is how platforms survive. It is overkill for a weekend prototype. The tax is coordination: contracts between modules, shared tracing, and an evaluation harness that can attribute errors to the right stage.

13. Advanced RAG

Not a single algorithm — a stack. Query rewrite, hybrid retrieval, reranking, memory, citation checks, and sometimes a correction loop, all wired together. This is what teams mean when they say “production RAG” after the first year.

Use it when a wrong answer has a dollar or reputational cost. Accept that you now have a distributed system, not a notebook. If you cannot explain which stage fixed a bad answer last week, the stack is too clever for your ops maturity.

14. Agentic RAG

The model is no longer a passenger. It plans: decide whether to search, which tool to call, whether the evidence is enough, whether to split the task. It may query a SQL warehouse, hit an API, re-read a PDF, then write. Retrieval is one tool among several.

This is the right shape for research-like work — due diligence, multi-source financial questions, long investigations. It is the wrong shape for “what is the refund window?” You will pay in latency, tracing complexity, and the need for hard guardrails so the agent does not wander.

How to choose without overbuilding

•        Start with Simple RAG plus decent chunking and a hybrid keyword+vector retriever. Measure groundedness and citation precision before you add loops.

•        Add memory only after users complain that follow-ups lose context.

•        Add Self-RAG or Corrective RAG when your error analysis shows “retrieved the wrong thing,” not “model phrased it badly.”

•        Add Graph RAG when answers require relationships the chunk store cannot see.

•        Add Multimodal RAG when the answer lives in a figure more often than in a paragraph.

•        Promote to Modular / Advanced when more than one team owns the pipeline.

•        Promote to Agentic only when the task is multi-step by nature and you can afford traces, budgets, and tool permissions.

A note on tools (without the vendor pitch)

Any of these patterns can sit on a fast hybrid search engine, a vector database, or a combination. Orchestration usually lives in LangChain, LangGraph, LlamaIndex, Haystack, or a thin custom graph. Embeddings and generators are interchangeable if your interfaces are clean — that is the whole point of Modular RAG.

What matters more than the logo is evaluation. Log the query, the retrieved IDs, the rerank order, and whether a human would have cited the same passages. Architecture debates end quickly when those numbers are on a dashboard.

Closing

RAG is not one product. It is a family of control loops around a language model. Naive and Simple RAG still ship the majority of useful assistants. The other twelve exist to fix specific failure modes: missing context, weak evidence, multimodality, relationships, and multi-step work.

Pick the cheapest loop that survives your eval set. Everything else is theater.