研究LoRA微调在文档问答系统中质量、延迟与资源的权衡，发现仅更新注意力的q/v投影最高效。
Analyzing Quality-Latency-Resource Trade-offs in a Technical Documentation RAG Assistant Using LoRA Adaptation
- 用LoRA微调生成器，只更新注意力的q/v投影层，性能最优
- 在Kubernetes文档上测试，3B/8B模型差异主要影响运行模式而非效率
- 结果具可复现性，代码数据已开源，适合系统优化研究者参考
我们研究基于官方Kubernetes文档的检索增强生成（RAG）系统中，使用低秩适配（LoRA）微调生成器时的质量-延迟-资源权衡。构建了包含5,144个问答对的手动验证基准，结合固定混合检索流水线（BGE-M3密集、BGE-M3原生稀疏、倒数排名融合、交叉编码器重排序）。在此基准上，对Llama-3.2-3B-Instruct和Llama-3.1-8B-Instruct进行20种LoRA配置的消融实验，评估指标包括逐标记F1、LLM判断的依据性与正确性（pass@4）、推理延迟、内存占用及训练成本，均报告自举95%置信区间。帕累托分析显示，仅作用于注意力模块的q和v投影的LoRA适配器始终占据前沿；3B与8B模型的选择主要决定系统运行范式。参数匹配对照实验进一步表明，q/v优势是结构性的，而非单纯由参数量决定。基准数据、选中的适配器及代码已公开于https://github.com/EugPal/rag-lora-tradeoffs。
原文摘要 · Abstract (English)
We study quality-latency-resource trade-offs in a documentation-grounded retrieval-augmented generation (RAG) system that uses Low-Rank Adaptation (LoRA) of the generator. We build a manually verified benchmark of 5,144 question-answer pairs over the official Kubernetes documentation and combine it with a fixed hybrid-retrieval pipeline (BGE-M3 dense, BGE-M3 native sparse, Reciprocal Rank Fusion, cross-encoder reranking). Over this benchmark we ablate 20 LoRA configurations on Llama-3.2-3B-Instruct and Llama-3.1-8B-Instruct across rank and target-module choices, and evaluate each on token-level F1, LLM-judged groundedness and correctness (pass@4), inference latency, inference memory, and training cost, all reported with bootstrap 95% confidence intervals. Pareto analysis shows that LoRA adapters acting only on the q and v attention projections consistently dominate the front, while the 3B/8B choice mainly defines operating regime. A param-matched control comparison further indicates that the q/v advantage is structural rather than purely parametric. The benchmark, selected adapters, and code are available at https://github.com/EugPal/rag-lora-tradeoffs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。