无需外部监督，语言模型可通过优化连贯性自我提升。
Self-Improvement as Coherence Optimization: A Theoretical Account
- 将自改进视为上下文到行为的最简可预测映射优化
- 理论证明连贯性优化等价于描述长度正则化，最优适用于半监督学习
- 解释无反馈自提升为何有效，并预测其适用边界
语言模型能否在无外部监督下提升准确性？辩论、自举和内部连贯性最大化等方法实现了这一看似神奇的效果，甚至达到黄金微调性能。然而其理论机制尚不清晰。本文表明，这些方法均是连贯性优化的特例：寻找最可压缩且联合可预测的上下文到行为映射。我们证明连贯性优化等价于描述长度正则化，并指出当正则化项来自预训练模型时，该方法在半监督学习中具有最优性。理论得到初步实验支持，解释了无反馈自改进的有效性，并预测其成功或失败的条件。
原文摘要 · Abstract (English)
Can language models improve their accuracy without external supervision? Methods such as debate, bootstrap, and internal coherence maximization achieve this surprising feat, even matching golden finetuning performance. Yet why they work remains theoretically unclear. We show that they are all special cases of coherence optimization: finding a context-to-behavior mapping that's most compressible and jointly predictable. We prove that coherence optimization is equivalent to description-length regularization, and that among all such regularization schemes, it is optimal for semi-supervised learning when the regularizer is derived from a pretrained model. Our theory, supported by preliminary experiments, explains why feedback-free self-improvement works and predicts when it should succeed or fail.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。