探索性建模的真正优势在于模型自由度,而非生成多样性。
Why the Third Axis Is Freedom
- 用多输出选择最接近样本的策略提升模型自由度
- 自由度越高,模型在分布外任务中表现越好,超越传统方法110%-500%
- 适合关注模型泛化能力与训练机制优化的研究者
生成式训练中,模型输出与示例对比并受惩罚。单输出模式下,一致答案更易胜出;而探索性建模(XM)每轮生成K个输出,仅更新最接近的,宣称探索是生成表达力的第三预训练轴。本文揭示该轴实为‘自由度’——即模型行为约束的薄弱程度。先前研究表明,自由度是函数属性而非形式属性,参数、架构、最小描述长度(MDL)和数据可变,但行为约束不变。理论证明最弱模型最易泛化,自由度选择在归纳实验中优于MDL达110%-500%。本文推导平均XM损失取决于候选项错过可接受区域的概率,探索使该概率升至幂次K。当K>1时,匹配概率随自由度上升。实验验证:前向XM中,增大K显著提升或饱和测量自由度,在上下文依赖目标下各值均提升;在候选池训练中,基于未标注父上下文的自由度选择器在30组测试中胜出29次。生成表达力仅是自由度的模式计数代理,忽略其结构带来的泛化意义。XM是手段,自由度是目的,以自由度为目标优化提升了模型在分布外场景的表现。
原文摘要 · Abstract (English)
In generative training, a model produces an output and is penalised for its difference from an example. With one output per comparison, a model that produces one common answer can outperform a model retaining a broader repertoire. Explorative Modeling (XM) produces $K$ outputs per comparison and updates on the closest, claiming exploration as a "third pretraining axis" associated with generative expressivity. Here I show the third axis is actually freedom, meaning the weakness of the constraint implied by a model's behaviour. Previous work showed freedom is a property of function rather than form. Parameters, architecture, minimum-description-length (MDL), and data can vary while the behavioural constraint remains unchanged. It was formally proved that weakest models are likeliest to generalise, and freedom selection beat MDL by 110-500\% in induction experiments. I prove average XM loss depends on the chance a candidate misses an acceptable region, with exploration raising miss probability to power $K$. For $K>1$, match probability rises with freedom. I then demonstrate empirically that XM optimises for freedom. In a Forward XM experiment, larger $K$ increased or saturated measured freedom, and increased freedom at every tested value under context-dependent targets. I trained XM candidate pools and compared validation selection with a freedom selector that read unlabelled parent contexts. Freedom won in 29 of 30 cases. Generative expressivity is a mode-count proxy for freedom, that discards the extension structure that gives freedom its generalisation significance. XM is a means, freedom an end, and selecting for freedom improved XM under distribution shift.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。