No matter how smart the big model is, it cannot cover the private knowledge within the enterprise. Retrieval Augmented Generation (RAG), through a "search first, generate later" architecture, injects external knowledge into the generation process, becoming the mainstream solution for enterprise knowledge base Q&A. But many teams simplify RAG - from demo to production, there are five key designs in between.
1、 Document processing: Splitting strategy determines answer quality
The original documents of the knowledge base are diverse: PDF, Word, tables, and slides. Chunking is the first process of RAG. The principle is' semantic completeness takes precedence over uniform length ': dividing by structural boundaries such as titles and paragraphs, rather than mechanically applying a one size fits all approach every 500 words. It is recommended to process table content by rows or blocks to avoid breaking the entire table into fragments during retrieval. The cleaning process is equally important, as header and footer, watermarks, and garbled characters can all contaminate the vector.
2、 Vectorization and Hybrid Retrieval
Pure vector retrieval is friendly to synonymous rewriting and semantically similar expressions, but it can easily fail to accurately match (model, number, person name). It is recommended to use a mixed mode of "vector search+keyword search" for the production environment, using inverted indexes to provide accurate queries, and then using a reordering model to merge and sort the two results. Although reordering adds one more model call, the improvement in answer accuracy is often immediate.
3、 Balance between recall and contextual window
Inserting too many fragments into prompt words can easily lead to the model getting lost in the middle, resulting in poorer answers. Practical experience is to prioritize the first few fragments that are most relevant to the problem and control the total token budget; At the same time, retain the source metadata of each fragment, so that the answers can be traced back.
4、 Reference tracing: making answers verifiable
In enterprise scenarios, 'model theory' does not equal 'company regulations'. Each conclusion output by RAG should be accompanied by an original citation, and users can click to return to the source document. This is both a compliance requirement and a key to building trust - users trust verifiable answers, not smooth illusions.
5、 Evaluation loop: Without evaluation, there can be no optimization
The launch of RAG is not the end. Establish an evaluation set covering typical problems, and regularly conduct regression tests using indicators such as "retrieval hit rate, answer accuracy, and citation accuracy"; Every time the segmentation strategy, embedding model, or reordering model is changed, the evaluation set must be run again before going online. RAG optimization without evaluation loop, mostly relying on intuition.
Summary: There is no silver bullet in the engineering of RAG. The core is to polish the five links of "document processing, retrieval strategy, context arrangement, citation tracing, and evaluation iteration" as a systematic engineering. Only by laying a solid foundation can enterprise knowledge base Q&A truly move from being able to answer to being able to answer correctly, accurately, and traceable.