用角色强化学习自动分配大模型，实现长文本处理的高效与低成本。
Role-RL: Online Long-Context Processing with Role Reinforcement Learning for Distinct LLMs in Their Optimal Roles
- 基于实际表现动态分配不同大模型在流水线中的角色。
- 在OPL-MINI数据集上达到93.2%平均召回率，成本降低79.4%。
- 适合需要实时处理长文本流的场景，如直播电商、新闻生成。
具备长上下文处理能力的大语言模型（LLMs）仍面临实现复杂性高、训练效率低和数据稀疏等问题。为此，本文提出一种在线长文本处理（OLP）新范式，适用于新闻自动播报、直播电商、爆款短视频等持续输入场景。面对海量模型中难以兼顾性能、成本与响应延迟的困境，我们设计了角色强化学习（Role-RL），根据各模型实际表现自动将其部署于OLP流水线中的最优角色。在自建的OLP-MINI数据集上进行大量实验，结果表明，采用Role-RL的OLP框架达成基准表现，平均召回率达93.2%，大模型使用成本降低79.4%。代码与数据集已公开：https://anonymous.4open.science/r/Role-RL。
原文摘要 · Abstract (English)
Large language models (LLMs) with long-context processing are still challenging because of their implementation complexity, training efficiency and data sparsity. To address this issue, a new paradigm named Online Long-context Processing (OLP) is proposed when we process a document of unlimited length, which typically occurs in the information reception and organization of diverse streaming media such as automated news reporting, live e-commerce, and viral short videos. Moreover, a dilemma was often encountered when we tried to select the most suitable LLM from a large number of LLMs amidst explosive growth aiming for outstanding performance, affordable prices, and short response delays. In view of this, we also develop Role Reinforcement Learning (Role-RL) to automatically deploy different LLMs in their respective roles within the OLP pipeline according to their actual performance. Extensive experiments are conducted on our OLP-MINI dataset and it is found that OLP with Role-RL framework achieves OLP benchmark with an average recall rate of 93.2% and the LLM cost saved by 79.4%. The code and dataset are publicly available at: https://anonymous.4open.science/r/Role-RL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。