arXiv:2607.18756cs.IRcs.CL2026-07

在8GB笔记本上实现政府机构本地化技术问答,零数据外泄

RAGAL: A Frugal, Fully Local Retrieval-Augmented Assistant for Technical Support at a Government Agency

  • 用混合稠密-稀疏检索+意图路由提升召回率
  • 微调嵌入模型后召回@10达85.0%,72分钟完成
  • 适合数据敏感、无云资源的政府或机构使用

公共机构拥有大量敏感文档与工单,无法外传,完全排除云端大模型。我们介绍RAGAL,一个为罗马尼亚农村投资融资局(AFIR)技术支持团队打造的本地化检索增强助手,运行于三项严苛约束下:零数据外泄(禁用外部API,含合成数据)、只读指令(仅生成草案,由人执行)、仅限一台8GB消费级笔记本作为开发训练设备。基于约2.5万段罗马尼亚语文本(15,073个已解决工单及内部规范文件),我们发现最有效投入是检索工程与检索器微调而非更大生成器:混合稠密-稀疏检索配合意图路由使内部评估准确率从62%升至81%;对bge-m3嵌入模型在真实工单数据上微调72分钟后,召回@10从0.663提升至0.850(MRR从0.489升至0.684)。我们发现单领域微调会悄然降低未触碰文档域的检索性能,通过构建分域评估集才被识别,并用本地生成查询(GenQ)修复。报告两个反直觉发现:PII掩码提升生成质量,结构化“锚点蒸馏”机制从构造上杜绝了SQL幻觉。提供可在8GB显存内完成全嵌入模型微调的可复现方案。由于零外泄也禁止云评判模型,我们提出替代:用744B参数模型在CPU上批处理运行,虽慢但可夜间执行,作为第二意见并量化其局限性。我们发布经脱敏的完整流程脚本,供面临类似数据本地化约束的机构使用。

原文摘要 · Abstract (English)

Public institutions hold large volumes of sensitive documents and support tickets that cannot leave the premises, ruling out cloud-hosted language models entirely. We report on RAGAL, a retrieval-augmented assistant for the technical-support team of AFIR, the Romanian Agency for Financing Rural Investments, built and operated under three hard constraints: zero data egress (no external API calls, even for synthetic data), a read-only mandate (the assistant drafts, humans execute), and a single 8 GB consumer laptop as the only development and training machine. Over a Romanian-language corpus of ~25,000 chunks -- 15,073 resolved support tickets and internal normative documents -- we show that the highest-leverage investments were retrieval engineering and retriever fine-tuning rather than a larger generator: hybrid dense-sparse retrieval with intent routing raised our internal evaluation from 62% to 81%, and fine-tuning the bge-m3 embedder on real ticket data improved recall@10 from 0.663 to 0.850 (MRR 0.489 to 0.684) after 72 minutes of training. We document a general pitfall: single-domain fine-tuning silently degraded retrieval on the untouched document domain below the stock baseline, detected only after building a per-domain evaluation set and repaired with locally generated queries (GenQ). We report two counter-intuitive findings -- PII masking improved generation quality, and a structural "anchor distillation" scheme made SQL hallucination impossible by construction -- along with a reproducible recipe for full embedder fine-tuning in 8 GB of VRAM. Finally, since zero egress also rules out a cloud judge, we describe a substitute: a 744B-parameter model run on CPU, too slow to serve interactively but affordable in overnight batch, used as a second opinion whose limits we quantify. We release the sanitized pipeline scripts for institutions facing similar data-locality constraints.

RAG本地部署数据安全检索增强

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。