通过动态自适应数据筛选，让大模型在长文本学习中持续进化。
PolicyLong: Towards On-Policy Context Extension
- 用当前模型反复筛选数据，实现训练分布随模型能力动态更新。
- 在128K上下文任务上比基线提升2.54分，越长越有效。
- 适合研究长文本生成与模型自我优化的开发者。
扩展大模型上下文窗口受限于高质量长上下文数据稀缺。现有方法通过信息论验证合成具备真实长程依赖的数据，选择能降低基础模型预测熵的上下文。但其使用固定模型进行单次离线构建，存在根本性非策略差距：静态筛选机制与模型能力演化不匹配，导致训练分布漂移。本文提出PolicyLong，将数据构建转向动态策略范式。通过迭代重执行数据筛选（熵计算、检索、验证），利用当前模型驱动筛选过程，确保训练分布跟踪模型能力演变，形成涌现的自适应课程。关键的是，正样本与难负样本均来自当前模型的熵分布，使模型学习内容与其需克服的挑战共同演化。在RULER、HELMET和LongBench-v2（Qwen2.5-3B）上的实验表明，PolicyLong始终优于EntropyLong和NExtLong，且在更长上下文下优势显著（如在RULER上128K处提升+2.54），验证了策略内数据演化的价值。
原文摘要 · Abstract (English)
Extending LLM context windows is hindered by scarce high-quality long-context data. Recent methods synthesize data with genuine long-range dependencies via information-theoretic verification, selecting contexts that reduce a base model's predictive entropy. However, their single-pass offline construction with a fixed model creates a fundamental off-policy gap: the static screening landscape misaligns with the model's evolving capabilities, causing the training distribution to drift. We propose PolicyLong, shifting data construction towards a dynamic on-policy paradigm. By iteratively re-executing data screening (entropy computation, retrieval, and verification) using the current model, PolicyLong ensures the training distribution tracks evolving capabilities, yielding an emergent self-curriculum. Crucially, both positive and hard negative contexts derive from the current model's entropy landscape, co-evolving what the model learns to exploit and resist. Experiments on RULER, HELMET, and LongBench-v2 (Qwen2.5-3B) show PolicyLong consistently outperforms EntropyLong and NExtLong, with gains growing at longer contexts (e.g., +2.54 at 128K on RULER), confirming the value of on-policy data evolution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。