通过移除前缀发现模型状态边界，实现无需训练的文本分块。
Right Reset: Chunking by Prefix Removal
- 基于移除前缀后隐藏态轨迹不变性，设计新分块方法
- 在去结构化文本中恢复47.7%原始记录，超基线一倍以上
- 适用于多语言模型，尤其适合无标注或弱结构数据
从因果语言模型中移除左侧上下文可揭示一种有效边界：模型对右侧相同标记的处理变化极小。本文提出前缀移除探测法，并引入右重置（Right Reset, RR），用于衡量右侧隐藏态轨迹的保留程度。通过动态规划将RR边缘得分转化为变长文本块。在去除分隔符与版式后的主题相似记录拼接文本上，RR恢复了47.7%的原始记录作为完整单元，远超25.9%的BGE嵌入边界基线，且该优势在渲染和OCR后依然存在。同层Qwen3-4B的被动评分及同规模指令模型的直接提示表现显著更差。在六种语言模型中，RR选中的切分点比未选候选边界的局部输出扰动更小。观察到的令牌似然比读出在某些架构中表现良好，表明核心贡献在于干预机制本身——当表面结构弱时，上下文依赖即可提供边界信号。
原文摘要 · Abstract (English)
Removing the left context from a causal language model reveals a useful kind of boundary: an edge where the model processes the same right-hand tokens with little change. We turn this observation into prefix-removal probing and introduce Right Reset (RR), which measures preservation of the right-hand hidden-state trajectory. A dynamic program converts RR edge scores into variable-length chunks. On flattened text formed by concatenating topically similar records after deleting their separators and layout, RR recovers 47.7% of the original records as clean units, versus 25.9% for a BGE embedding boundary, the strongest tested conventional baseline without task-specific model training. The gain persists after rendering and OCR. Passive scores from the same Qwen3-4B layer and direct prompting of a same-scale instruction model perform substantially worse on flattened records. Across six language models, RR-selected cuts also undergo consistently less local output disruption than unselected candidate edges. An observed-token likelihood-ratio readout is competitive in some architectures, indicating that the central contribution is the intervention: context dependence itself can provide a boundary signal when surface structure is weak.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。