RAG has become the default reflex whenever a company wants to connect AI to its documents. The demo is fast: a PDF folder, an embedding, a question, a convincing answer. In production the same architecture hits obsolete documents, access rights, duplicates, and no criterion for saying an answer is acceptable.
Sources, freshness, citations
What actually works starts before the model. Which sources are allowed. Who maintains them. How often they are re-indexed. An answer without a usable citation is not a business tool, it is a conversation. A citation to a stale document is worse: it lends confidence to an error. The same gap between demo and production appears in document extraction.
Access rights complicate the picture further. An employee should not obtain, through RAG, information they would not have had otherwise. That means propagating permissions into retrieval, not only at ingestion time. Many demos ignore this because they run on an open corpus. Real IT landscapes are not open.
Evaluate in order to improve
Evaluation remains the other blind spot. Without business question sets, without a refusal rate when context is insufficient, without measuring human intervention, you cannot tell whether the system is improving. “It looks right” in a steering committee is not a metric. Teams that make progress build a small truth set and keep it alive across versions. Without a semantic layer, retrieval also pulls from contradictory definitions.
Production RAG looks less like a chatbot and more like a pipeline: governed ingestion, constrained retrieval, generation with citation, logging, correction loop. If you are still at the impressive-demo stage, the useful question is not which model to pick. It is which sources and truth criteria you are willing to defend. To frame that pipeline, see also our AI offerings.
