The claim that roughly 70% of enterprise RAG pilots fail to reach production has been circulating in practitioner circles long enough that it’s worth examining the failure modes specifically, not just the statistic.
The retrieval architecture itself — embedding quality, chunk size, index construction, query rewriting — accounts for fewer failures than most technical coverage suggests. Teams that get to the retrieval architecture stage have usually built something that works well enough to proceed. The failures happen before and after.
Before: knowledge base quality. RAG systems perform as well as the documents you feed them. Enterprise knowledge bases are consistently worse than pilot builders assume — out-of-date content, inconsistent formatting, documents that were designed for humans to skim rather than for models to retrieve from, and content that assumes context that only employees with institutional memory actually have. Getting the knowledge base into a state where retrieval works reliably is a content operations problem, not an engineering problem, and many organizations don’t have a clear owner for it.
After: evaluation. The failure mode here is shipping a system that works on the demo but not on the distribution of actual user queries. Teams that build good retrieval but don’t build rigorous evaluation pipelines discover their system’s failure modes in production, where the cost is user trust rather than test failures.
The organizational failure mode: RAG pilots often succeed because one engineer built something clever and ran it past the team. Production deployment requires ownership of the knowledge base, ongoing evaluation, a process for handling retrieval failures, and someone accountable when the AI gives a wrong answer. Organizations that don’t have that structure before the pilot often don’t build it before the production decision.
What actually makes it to production: systems with a clear knowledge base owner, a defined scope of queries the system is expected to handle, and an evaluation suite that covers the failure modes that matter for the use case.