Advanced RAG Patterns

Beyond basic retrieval: HyDE, Graph RAG, Agentic RAG, Multi-modal, and production patterns.

HyDE (Hypothetical Document Embeddings)

Concept

Generate a hypothetical answer → embed it → retrieve similar real documents.

from llama_index.core.retrievers import HyDERetriever

hyde_retriever = HyDERetriever(
    vector_retriever=vector_retriever,
    llm=llm,
    include_original=True,  # Also embed original query
    num_hypothetical=3  # Generate multiple
)

How It Works

Query: "What causes Alzheimer's?"
    ↓
LLM generates: "Alzheimer's is caused by amyloid plaques and tau tangles..."
    ↓
Embed hypothetical answer → Retrieve similar real docs
    ↓
Use real docs for grounded generation

When to Use

• Queries where user lacks vocabulary
• Abstract/conceptual questions
• Domain-specific terminology gaps
• Low recall with standard retrieval

Graph RAG

Knowledge Graph Construction

from llama_index.core import KnowledgeGraphIndex
from llama_index.graph_stores import Neo4jGraphStore
from llama_index.core import SimpleDirectoryReader

graph_store = Neo4jGraphStore(
    username="neo4j",
    password="...",
    url="bolt://localhost:7687"
)

documents = SimpleDirectoryReader("./data").load_data()

kg_index = KnowledgeGraphIndex.from_documents(
    documents,
    graph_store=graph_store,
    max_triplets_per_chunk=10,
    include_embeddings=True  # Hybrid: graph + vector
)

Querying

# Graph traversal query
query_engine = kg_index.as_query_engine(
    include_text=True,
    retriever_mode="keyword",  # or "embedding"
    response_mode="tree_summarize"
)

response = query_engine.query(
    "What proteins interact with BRCA1 in breast cancer?"
)

Graph RAG Patterns

Advanced: Graph RAG with LLM

from llama_index.core.query_engine import GraphRAGQueryEngine

query_engine = GraphRAGQueryEngine(
    graph_store=graph_store,
    llm=llm,
    similarity_top_k=10,
    # Uses LLM to generate Cypher queries
    # or natural language traversal
)

Agentic RAG

Agent as Retriever

from llama_index.agent import ReActAgent
from llama_index.core.tools import QueryEngineTool

rag_tool = QueryEngineTool.from_defaults(
    query_engine=query_engine,
    name="knowledge_base",
    description="Search internal knowledge base"
)

agent = ReActAgent.from_tools(
    tools=[rag_tool, web_search, calculator],
    llm=llm,
    verbose=True
)

# Agent decides when/how to retrieve
response = agent.query(
    "Find the latest research on transformer efficiency, "
    "compare with our internal benchmarks, and summarize"
)

Multi-Step Retrieval

from llama_index.agent import FunctionCallingAgent

agent = FunctionCallingAgent.from_tools(
    tools=[
        QueryEngineTool(query_engine, name="docs"),
        QueryEngineTool(web_engine, name="web"),
        FunctionTool.from_defaults(fn=search_code, name="code_search")
    ],
    llm=llm,
    max_iterations=10
)

# Agent plans multi-hop retrieval
response = agent.query(
    "Find our design doc for auth system, "
    "check if it mentions OAuth2, "
    "and compare with current implementation"
)

Retrieval Planning

from llama_index.core.workflow import Workflow, StartEvent, StopEvent, step

class RetrievalPlanWorkflow(Workflow):
    @step
    async def plan(self, ctx: Context, ev: StartEvent) -> RetrievalPlanEvent:
        plan = await llm.acomplete(f"""
        Create retrieval plan for: {ev.query}
        Available sources: internal_docs, web, code, api_docs
        Return JSON with queries per source.
        """)
        return RetrievalPlanEvent(plan=parse_json(plan.text))

    @step
    async def execute(self, ctx: Context, ev: RetrievalPlanEvent) -> RetrievalResultEvent:
        results = {}
        for source, queries in ev.plan.items():
            for q in queries:
                results[f"{source}:{q}"] = await self.retrieve(source, q)
        return RetrievalResultEvent(results=results)

    @step
    async def synthesize(self, ctx: Context, ev: RetrievalResultEvent) -> StopEvent:
        return StopEvent(result=await self.synthesize(ev.results))

Multi-Modal RAG

Vision + Text

from llama_index.core import MultiModalVectorStoreIndex
from llama_index.core import SimpleDirectoryReader
from llama_index.multi_modal_llms import OpenAIMultiModal

# Load images + text
documents = SimpleDirectoryReader(
    "./data",
    required_exts=[".pdf", ".png", ".jpg"]
).load_data()

# Multi-modal index
index = MultiModalVectorStoreIndex.from_documents(
    documents,
    embed_model=OpenAIMultiModal(model="gpt-4o")
)

query_engine = index.as_query_engine(
    llm=OpenAIMultiModal(model="gpt-4o"),
    similarity_top_k=5
)

response = query_engine.query(
    "What does the architecture diagram show?",
    image_paths=["diagram.png"]
)

Table/Chart Understanding

from llama_index.core import TableIndex

table_index = TableIndex.from_documents(documents)

query_engine = table_index.as_query_engine(
    llm=llm,
    response_mode="tree_summarize"
)

response = query_engine.query(
    "What's the revenue trend in Q3 vs Q4?"
)

Long-Context RAG

Context Window Optimization

from llama_index.core.postprocessor import LongContextReorder

query_engine = RetrieverQueryEngine.from_args(
    retriever=retriever,
    node_postprocessors=[
        LongContextReorder()  # Put most relevant in middle
    ]
)

Recursive Summarization

from llama_index.core.query_engine import SummaryQueryEngine

summary_engine = SummaryQueryEngine(
    retriever=retriever,
    summary_query="Summarize key findings",
    detail_query="Provide details on {topic}"
)

Hierarchical Retrieval

# Document → Section → Chunk
from llama_index.core.retrievers import RecursiveRetriever

retriever = RecursiveRetriever(
    "document",
    retriever_dict={
        "document": document_retriever,    # Top-level
        "section": section_retriever,      # Mid-level
        "chunk": chunk_retriever           # Leaf
    }
)

Production Patterns

Query Understanding

class QueryProcessor:
    async def process(self, query: str) -> ProcessedQuery:
        # 1. Decompose
        subqueries = await self.decompose(query)

        # 2. Rewrite for retrieval
        rewritten = await self.rewrite_for_retrieval(subqueries)

        # 3. Identify required sources
        sources = self.identify_sources(rewritten)

        return ProcessedQuery(
            original=query,
            subqueries=rewritten,
            sources=sources
        )

Adaptive Retrieval

class AdaptiveRetriever:
    async def retrieve(self, query: ProcessedQuery) -> list[Node]:
        strategy = self.select_strategy(query)

        if strategy == "hybrid":
            return await self.hybrid_retrieve(query)
        elif strategy == "hyde":
            return await self.hyde_retrieve(query)
        elif strategy == "graph":
            return await self.graph_retrieve(query)
        else:
            return await self.default_retrieve(query)

Feedback Loop

class RAGWithFeedback:
    def __init__(self):
        self.feedback_store = FeedbackStore()

    async def query(self, query: str) -> Response:
        response = await self.rag.query(query)

        # Log for feedback
        await self.log_query(query, response)

        return response

    async def collect_feedback(self, query_id: str, rating: int, correction: str):
        await self.feedback_store.save(query_id, rating, correction)

        # Retrain retriever/reranker periodically
        if len(self.feedback_store) % 100 == 0:
            await self.retrain()

Evaluation & Monitoring

RAG-Specific Metrics

A/B Testing

class RAGABTest:
    async def query(self, query: str, user_id: str) -> Response:
        variant = self.assign_variant(user_id)  # "baseline" | "hyde" | "graph"

        if variant == "baseline":
            return await self.baseline_rag.query(query)
        elif variant == "hyde":
            return await self.hyde_rag.query(query)
        elif variant == "graph":
            return await self.graph_rag.query(query)

Related Notes

• RAG Fundamentals — Basic RAG pipeline
• Memory Systems — RAG as long-term memory
• Context Window Management — Context optimization
• Knowledge Graphs — Graph RAG deep dive
• Multimodal Context — Multimodal RAG
• LlamaIndex Agents — LlamaIndex implementation
• LangGraph — Agentic RAG with LangGraph
• Agent Benchmarks — RAG benchmarks
• Research Directions — Future RAG directions

Sources: "HyDE" (Gao et al., 2022), "Graph RAG" (Edge et al., 2024), "Agentic RAG" (LlamaIndex), "LongLoRA" (Chen et al.), "Recursive Retrieval" (LlamaIndex), production RAG post-mortems (Notion, GitHub, Stripe), RAG evaluation papers.