用轻量强化学习优化文档选择，让本地部署的RAG更快更准。
SRAS: A Lightweight Reinforcement Learning-based Document Selector for Edge-Native RAG Pipelines
- 通过强化学习训练小型策略模型，动态选择最优文档
- 在CPU上<1秒延迟下，SQuAD测试达0.8546的BERTScore
- 无需调参即可适配真实数据，适合边缘设备部署
检索增强生成（RAG）系统常依赖固定的前k个文档选择机制，忽视生成质量且计算开销大。本文提出SRAS（稀疏奖励感知选择器），一种基于强化学习的轻量级文档选择器，专为边缘原生RAG设计。与以往依赖大内存和高延迟预算的强化学习检索器不同，SRAS采用近端策略优化（PPO）训练出约0.76MB的小型策略模型，其奖励信号融合松弛F1与BERTScore。该方法在严格令牌与计算约束下运行，可在CPU上保持<1秒延迟。SRAS在合成问答基准上优于监督与随机选择器，并在真实数据集SQuAD v2上实现0.8546的BERTScore F1，无需领域特定调参。这是首个证明强化学习文档选择可实现超轻量化、低延迟且适用于本地RAG系统的成果。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) systems often rely on fixed top-k document selection mechanisms that ignore downstream generation quality and impose computational overheads. We propose SRAS (Sparse Reward-Aware Selector), a lightweight document selector trained via reinforcement learning (RL) for edge-native RAG deployment. Unlike prior RL-based retrievers that assume large memory and latency budgets, SRAS learns a compact (~0.76MB) policy using Proximal Policy Optimization (PPO), guided by a hybrid reward signal combining Relaxed F1 and BERTScore. Our method operates under tight token and compute constraints, maintaining <1s latency on CPU. SRAS outperforms supervised and random selectors on a synthetic QA benchmark, and generalizes to real-world data, achieving BERTScore F1 of 0.8546 on SQuAD v2 without domain-specific tuning. This work is the first to demonstrate that RL-based document selection can be made ultra-lightweight, latency-aware, and effective for on-device RAG pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。