用强化学习生成边界，让小模型高效分割复杂文本
BoundRL: Efficient Structured Text Segmentation through Reinforced Boundary Generation
- 只生成段落起始标记，通过定位原文本重建内容，减少90%输出
- 小模型（1.7B参数）在复杂任务上超越大模型的提示工程方法
- 结合可验证奖励与边界扰动，防止熵坍塌，提升分割质量
结构化文本指包含代码片段、占位符等非纯文本元素的文本。这类文本需分割为语义上连贯的组件，传统句子级分割方法难以应对。为此，我们提出BoundRL，一种联合进行高效标记级文本分割与标签预测的新方法。该方法不生成每个片段的完整文本，而是仅生成起始标记，并通过在原文中定位这些标记来重构完整内容，使输出标记数减少90%，显著降低幻觉。为训练边界生成模型，BoundRL采用可验证奖励的强化学习（RLVR），联合优化文档重建保真度与语义一致性。同时，通过扰动段落边界和标签构建中间候选解，作为通往更优解的阶梯，缓解熵坍塌问题。实验表明，仅1.7B参数的小语言模型在复杂提示任务上，性能优于使用更大模型的少样本提示方法，以及SFT和标准RLVR基线。
原文摘要 · Abstract (English)
Structured texts refer to texts containing structured elements beyond plain texts, such as code snippets and placeholders. Such structured texts increasingly require segmentation into semantically meaningful components, which cannot be effectively handled by conventional sentence-level segmentation methods. To address this, we propose BoundRL, a novel approach that jointly performs efficient token-level text segmentation and label prediction for long structured texts. Instead of generating full texts for each segment, it generates only starting tokens and reconstructs the complete texts by locating these tokens within the original texts, thereby reducing output tokens by 90% and minimizing hallucination. To train the models for the boundary generation, BoundRL~performs reinforcement learning with verifiable rewards (RLVR) that jointly optimizes document reconstruction fidelity and semantic alignment. It further mitigates entropy collapse by constructing intermediate candidates by perturbing segment boundaries and labels to create stepping stones toward higher-quality solutions. Experiments show that BoundRL enables small language models (1.7B parameters) to outperform few-shot prompting with much larger models as well as SFT and standard RLVR baselines on complex prompts used for LLM applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。