Every team that builds a RAG prototype gets the same first impression: this is straightforward. You embed some documents, store them in a vector database, retrieve the top-k results, and feed them into an LLM. The demo works. Stakeholders are impressed. Then you try to ship it to production and discover the gap between a working notebook and a reliable system is enormous.
We have deployed RAG systems for enterprise clients across legal document search, internal knowledge bases, customer support automation, and compliance workflows. This post captures the architecture patterns and engineering decisions that separate production RAG from prototype RAG — the lessons we learned by breaking things in real deployments.
Why Naive RAG Fails in Production
The basic RAG pattern — embed query, retrieve top-k chunks, generate answer — has failure modes that only reveal themselves at scale with real users:
- Retrieval misses on short queries. A user types "refund policy" and your system returns chunks about "policy updates" and "refund processing timelines" but misses the actual refund policy document because the embedding similarity score is marginal.
- Context window pollution. You retrieve 10 chunks and 7 of them are irrelevant. The LLM now has to reason through noise, which degrades answer quality and increases hallucination rates.
- Chunking artifacts. A critical piece of information spans two chunks. Neither chunk alone contains enough context for the LLM to generate a correct answer. The user gets a confidently wrong response.
- Latency compounding. Embedding the query takes 200ms. Vector search takes 150ms. Reranking takes 300ms. LLM generation takes 2s. Your P95 latency is now 4 seconds, and users drop off.
- No feedback signal. Without evaluation infrastructure, you cannot measure whether retrieval quality is improving or degrading as you add documents. You are flying blind.
These are not edge cases. They are the default experience when you move from a curated demo dataset to production corpora with inconsistent formatting, domain-specific terminology, and ambiguous user queries.
Our Architecture: The Production RAG Stack
After iterating across multiple deployments, we converged on an architecture that handles real-world complexity. Here is the high-level stack:
User Query
|
v
[FastAPI Gateway] --- auth, rate limiting, query preprocessing
|
v
[Hybrid Retrieval]
|--- Dense: pgvector (OpenAI text-embedding-3-large)
|--- Sparse: BM25 (Elasticsearch / custom index)
|
v
[Reciprocal Rank Fusion] --- merge dense + sparse results
|
v
[Cross-Encoder Reranker] --- ms-marco-MiniLM-L-12-v2
|
v
[Context Assembly] --- dedup, ordering, citation mapping
|
v
[LLM Generation] --- GPT-4o with structured output
|
v
[Response + Citations] --- traceable back to source chunks
Document Ingestion and Chunking
Chunking strategy is the single most impactful decision in a RAG system, yet it gets the least attention during prototyping. We have moved away from naive fixed-size chunking entirely.
Our approach uses hierarchical chunking with overlap:
from langchain.text_splitter import RecursiveCharacterTextSplitter
# Parent chunks for context preservation
parent_splitter = RecursiveCharacterTextSplitter(
chunk_size=2000,
chunk_overlap=200,
separators=["\n\n", "\n", ". ", " "]
)
# Child chunks for precise retrieval
child_splitter = RecursiveCharacterTextSplitter(
chunk_size=400,
chunk_overlap=50,
separators=["\n\n", "\n", ". ", " "]
)
# We retrieve on child chunks but expand to parent
# chunks before sending to the LLM
def retrieve_with_parent_context(query: str, top_k: int = 5):
child_results = vector_store.similarity_search(query, k=top_k * 3)
parent_ids = set(r.metadata["parent_id"] for r in child_results[:top_k])
return [parent_store[pid] for pid in parent_ids]
The principle: retrieve on small, precise chunks (400 tokens) but expand to larger parent chunks (2000 tokens) before feeding into the LLM. This gives you retrieval precision without sacrificing the context the model needs to generate a coherent answer.
For structured documents (contracts, technical manuals, compliance docs), we also apply document-aware chunking that respects section boundaries, headers, and table structures. A chunk should never split a table row or break mid-paragraph unless the paragraph is unusually long.
Hybrid Retrieval: Dense + Sparse
Purely semantic search (dense vectors) fails on exact-match queries. A user searching for "SOC-2 Type II" needs lexical matching, not semantic similarity. Purely keyword search (BM25) fails on paraphrased or conceptual queries. You need both.
from fastapi import FastAPI, Depends
from pydantic import BaseModel
import numpy as np
app = FastAPI()
class RetrievalRequest(BaseModel):
query: str
top_k: int = 10
alpha: float = 0.6 # weight for dense vs sparse
@app.post("/retrieve")
async def hybrid_retrieve(req: RetrievalRequest):
# Dense retrieval via pgvector
query_embedding = await embed_query(req.query)
dense_results = await pgvector_search(
query_embedding, k=req.top_k * 2
)
# Sparse retrieval via BM25
sparse_results = await bm25_search(
req.query, k=req.top_k * 2
)
# Reciprocal Rank Fusion
fused = reciprocal_rank_fusion(
[dense_results, sparse_results],
weights=[req.alpha, 1 - req.alpha],
k=60
)
return fused[:req.top_k]
def reciprocal_rank_fusion(result_lists, weights, k=60):
"""Merge multiple ranked lists using RRF."""
scores = {}
for results, weight in zip(result_lists, weights):
for rank, doc in enumerate(results):
doc_id = doc.metadata["id"]
if doc_id not in scores:
scores[doc_id] = {"doc": doc, "score": 0.0}
scores[doc_id]["score"] += weight / (k + rank + 1)
ranked = sorted(scores.values(), key=lambda x: x["score"], reverse=True)
return [item["doc"] for item in ranked]
The alpha parameter controls the balance between dense and sparse. We default to 0.6 (favoring dense) but expose it as a tunable parameter per deployment. Some corpora — particularly those with heavy acronyms or product codes — need alpha closer to 0.4.
Reranking: The Critical Middle Layer
Initial retrieval is recall-optimized. You cast a wide net. The reranker is precision-optimized — it takes the top 20-30 candidates and re-scores them using a cross-encoder that sees query and document together.
This is the single highest-ROI addition to any RAG pipeline. In our measurements, adding a reranker consistently improves answer relevance by 15-25% (measured by RAGAS faithfulness) with only 200-400ms of added latency. For most enterprise use cases, that tradeoff is trivially worth it.
We use the cross-encoder/ms-marco-MiniLM-L-12-v2 model for reranking. It is small enough to run on CPU with acceptable latency, and the quality improvement over no-reranking is dramatic. For latency-sensitive deployments, we run it on a dedicated GPU instance and batch requests.
Multi-Scenario Retrieval Testing
In VLSI physical design, engineers run MCMM (Multi-Corner Multi-Mode) analysis to verify that a chip works across all operating conditions — fast/slow corners, different voltages, different temperature extremes. The equivalent in RAG is testing your retrieval across the full distribution of query types your system will encounter.
We define four retrieval scenarios that every deployment must pass:
- Short, specific queries (2-4 words) — "refund policy," "API rate limits," "SOC-2 compliance." These test lexical matching and metadata filtering.
- Long, contextual queries (20+ words) — "What is the process for requesting an extension on a contract renewal when the original terms included an automatic rollover clause?" These test semantic understanding and chunk boundary handling.
- Ambiguous queries — "how does billing work" (which billing? subscription? usage-based? invoicing?). These test whether the system can surface multiple relevant contexts or ask for clarification.
- Out-of-domain queries — questions the knowledge base cannot answer. These test whether the system gracefully declines rather than hallucinating an answer from tangentially related content.
We build golden datasets for each scenario — typically 50-100 query-answer pairs per category — and run automated evaluation on every pipeline change. If retrieval recall drops on any scenario after a configuration change, the change does not ship.
# retrieval_eval.py — scenario-based retrieval testing
import json
from ragas import evaluate
from ragas.metrics import context_recall, context_precision
from datasets import Dataset
def load_golden_dataset(scenario: str) -> Dataset:
with open(f"eval/golden_{scenario}.json") as f:
data = json.load(f)
return Dataset.from_dict({
"question": [d["query"] for d in data],
"ground_truth": [d["expected_answer"] for d in data],
"contexts": [retrieve(d["query"]) for d in data],
"answer": [generate(d["query"]) for d in data],
})
scenarios = ["short_specific", "long_contextual", "ambiguous", "out_of_domain"]
for scenario in scenarios:
ds = load_golden_dataset(scenario)
result = evaluate(ds, metrics=[context_recall, context_precision])
print(f"{scenario}: recall={result['context_recall']:.3f}, "
f"precision={result['context_precision']:.3f}")
# Gate: fail if recall drops below threshold
assert result["context_recall"] >= 0.85, (
f"Retrieval recall for {scenario} below threshold"
)
Evaluation: Measuring What Matters
You cannot improve what you cannot measure. We use RAGAS as the backbone of our evaluation framework, supplemented by human feedback loops for cases where automated metrics diverge from perceived quality.
The metrics we track in production:
- Context Recall — what fraction of the ground-truth answer is supported by retrieved contexts? If this drops, your retrieval is missing relevant documents.
- Context Precision — of the retrieved contexts, what fraction is actually relevant? Low precision means your LLM is swimming in noise.
- Faithfulness — is the generated answer supported by the retrieved contexts? Low faithfulness means hallucination is creeping in.
- Answer Relevancy — does the answer actually address the user's question? Orthogonal to faithfulness — an answer can be faithful to context but miss the user's intent entirely.
We run RAGAS evaluation nightly against a held-out test set and alert when any metric drops more than 5% from the rolling baseline. This catches regressions from new document ingestion, embedding model changes, or infrastructure issues before they affect users.
Human Feedback Loops
Automated metrics get you 80% of the way. The remaining 20% requires human signal. We instrument every RAG response with thumbs up/down feedback and a free-text "what went wrong" field. This feeds into a weekly review cycle where we:
- Identify the top failure patterns (usually 3-5 categories account for 80% of negative feedback)
- Add failing examples to our golden evaluation sets
- Trace failures back to retrieval (wrong chunks), generation (LLM hallucination), or data quality (source document is outdated/wrong)
- Ship targeted fixes and verify improvement in the next evaluation run
LangSmith provides the tracing infrastructure for this. Every query flows through LangSmith with full observability — we can see exactly which chunks were retrieved, what the reranker scored them, and how the LLM used them in generation. When a user reports a bad answer, we can reconstruct the entire pipeline execution in seconds.
Production Concerns
Latency Budget
Enterprise users expect sub-3-second responses for knowledge retrieval. Our latency budget breaks down as:
- Query embedding: 80-150ms (OpenAI API, with connection pooling)
- Hybrid retrieval: 50-120ms (pgvector + BM25 in parallel)
- Reranking: 200-400ms (cross-encoder, batched)
- LLM generation: 1-2.5s (GPT-4o, streaming)
Total P95: 2.5-3.2 seconds. We stream the LLM response so users see tokens appearing within 500ms of their query, even though generation is not complete. This dramatically improves perceived latency.
Caching Strategy
We implement two-tier caching:
- Embedding cache: If we have seen this exact query before, skip the embedding API call. Redis with a 24-hour TTL. Saves 100-150ms and reduces OpenAI costs by 30-40% on corpora with repetitive query patterns.
- Semantic cache: If the query is semantically similar (cosine > 0.97) to a previously answered query, return the cached response directly. This handles paraphrases — "what's the refund policy" and "how do refunds work" hit the same cache entry. Aggressive TTL (1 hour) to avoid stale answers.
Cost Management
At enterprise scale, RAG costs compound quickly. A single query touches: embedding API (input tokens), vector DB compute, reranker inference, and LLM generation (input + output tokens). At 10,000 queries/day, this adds up.
Our cost control levers:
- Embedding model selection:
text-embedding-3-smallis 5x cheaper thantext-embedding-3-largeand performs within 2-3% on most benchmarks. We default to small and only upgrade for deployments where retrieval quality metrics demand it. - Aggressive chunk deduplication: Before sending contexts to the LLM, we deduplicate overlapping content. This can reduce input tokens by 20-40%.
- Tiered generation: Simple factual queries go to GPT-4o-mini. Complex reasoning queries route to GPT-4o. Classification happens at the retrieval stage based on query complexity and retrieved context diversity.
Guardrails and Citation Tracking
Production RAG systems need guardrails at multiple levels:
- Input guardrails: Detect and reject prompt injection attempts, off-topic queries, and PII in user input before it reaches the retrieval pipeline.
- Output guardrails: Validate that the generated response is grounded in retrieved contexts. If the LLM generates a claim not traceable to any source chunk, we either flag it or strip it from the response.
- Citation enforcement: Every factual claim in the response maps to a specific source chunk with document ID, page number, and section header. Users can click through to verify. This is not optional for enterprise deployments — it is a trust requirement.
from pydantic import BaseModel
from typing import List
class Citation(BaseModel):
text: str
source_doc: str
chunk_id: str
page: int | None = None
section: str | None = None
class RAGResponse(BaseModel):
answer: str
citations: List[Citation]
confidence: float # 0-1, based on retrieval scores
retrieval_latency_ms: int
total_latency_ms: int
# We use structured output to enforce citation format
# LLM must output in this schema — no ungrounded claims
response = await openai_client.beta.chat.completions.parse(
model="gpt-4o",
messages=[
{"role": "system", "content": SYSTEM_PROMPT_WITH_CITATION_RULES},
{"role": "user", "content": assembled_prompt},
],
response_format=RAGResponse,
)
The Tech Stack
For reference, here is the complete stack we use across production RAG deployments:
- API layer: Python + FastAPI (async, connection pooling, middleware for auth/logging)
- Vector store: PostgreSQL + pgvector (HNSW indexes, cosine similarity)
- Sparse retrieval: Elasticsearch (BM25) or custom inverted index for smaller deployments
- Embeddings: OpenAI text-embedding-3-large (or small, per deployment)
- Reranker: cross-encoder/ms-marco-MiniLM-L-12-v2 (self-hosted)
- Orchestration: LangChain (LCEL chains, retriever abstractions)
- Generation: OpenAI GPT-4o / GPT-4o-mini (tiered by query complexity)
- Evaluation: RAGAS (automated metrics), LangSmith (tracing + observability)
- Caching: Redis (embedding cache + semantic cache)
- Infrastructure: AWS (ECS/Fargate, RDS for pgvector, ElastiCache)
- Monitoring: LangSmith traces + custom Datadog dashboards for latency/cost/quality
Lessons Learned
After shipping multiple production RAG systems, these are the patterns we keep returning to:
- Chunking matters more than model choice. Switching from GPT-4 to GPT-4o gives you marginal improvement. Fixing your chunking strategy to respect document structure gives you 20-30% improvement in answer quality.
- Evaluation first, optimization second. If you cannot measure retrieval quality, you cannot improve it. Set up RAGAS and golden datasets before you start tuning parameters.
- Hybrid retrieval is not optional. Every production deployment we have done benefits from combining dense and sparse retrieval. The failure modes are complementary — dense misses exact matches, sparse misses semantic similarity.
- Reranking is the highest-ROI intervention. Adding a cross-encoder reranker takes a day to implement and immediately improves answer quality. Do it before anything else.
- Source data quality is the ceiling. No amount of retrieval sophistication compensates for poorly formatted, outdated, or contradictory source documents. Budget time for data cleaning and establish ongoing data quality processes.
- Latency perception matters as much as latency reality. Streaming responses and showing retrieval progress indicators make a 3-second system feel faster than a 2-second system that shows a blank screen.
RAG is not a solved problem — the field moves quickly, and production requirements vary significantly across domains. But the architecture patterns above give you a solid foundation that handles real-world complexity rather than breaking silently on the first query that deviates from your demo dataset.