用代码生成动态信息演变数据,让大模型练出真实推理能力。
EvolveScaler: Synthesizing Information-Evolution Contexts via Executable State Machines and Natural-Language Rendering

- 先写代码定义状态变化规则,再转成自然语言对话历史
- 生成近3.5万条带参考答案的长序列推理数据,每例最多1200事件
- 适合训练和测试大模型在复杂演变场景下的长期推理能力
在持续交互中,长上下文可能描述一个动态演变过程:后续事件可修改或撤销早期信息,改变有效内容与推论结果。我们称此为信息演变(IE)。解决IE需识别有效记录、按序应用更新,并从事件历史重构查询相关状态。现有文本优先的生成流程使数据难以验证,因状态转移与答案逻辑隐含其中。我们提出EvolveScaler,一种代码驱动的框架:先以人工编写的操作规范定义状态转移、记录有效性、难度控制与可执行的答案逻辑;再由强语言模型从中生成自包含模拟器。运行经验证的模拟器可生成自然语言多轮事件历史,确定性重放则计算参考答案与原子检查清单。我们在五个难度等级上构建了117个任务原型和159个最终问题算子,每实例事件数约7至1200,共生成约3.51万条训练样本与585个验证实例。在very_long层级,最强模型达到平均59.3% avg@5,六种模型低于10%。在6000个EvolveScaler样本上训练内部A3B模型,在八个独立分布外基准上均优于基线,平均提升5.25分。结果表明,代码驱动的IE合成能提供挑战性评估与可迁移的训练监督。
原文摘要 · Abstract (English)
In persistent interactions, long contexts may encode an evolving process rather than a fixed record: later events can revise or revoke earlier information, changing what remains valid and what conclusions follow. We call this setting information evolution (IE). Solving IE requires identifying valid records, applying updates in order, and reconstructing the query-relevant state from the event history. Existing text-first synthesis pipelines make such data difficult to verify because state transitions and answer logic remain implicit. We introduce EvolveScaler, a code-driven framework that defines information evolution before rendering it as natural language. Human-authored operational specifications define state transitions, record validity, difficulty controls, and executable answer logic; a strong LLM then synthesizes a self-contained simulator from each specification. Executing validated simulators produces natural-language multi-turn event histories, while deterministic replay computes reference answers and atomic checklists. We instantiate EvolveScaler with 117 task prototypes and 159 final-question operators across five difficulty levels spanning approximately 7 to 1,200 events per instance, yielding about 35,100 training examples and 585 validated evaluation instances. On the very_long tier, the strongest model reaches 59.3% avg@5, while six models score below 10%. Training an internal A3B model on 6,000 EvolveScaler examples improves performance over its base checkpoint on all eight independently constructed out-of-distribution benchmarks, with a 5.25-point average gain. These results show that code-driven IE synthesis provides both challenging evaluation and transferable training supervision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。