模型因数据碎片化训练产生意外行为，工具可实时发现并定位根源。
Chunky Post-Training: Data Driven Failures of Generalization
- 通过黑箱分析识别训练数据中的隐含模式导致的异常行为。
- 在多个前沿模型中发现因数据分布不均引发的判断偏差，如格式敏感性错误。
- 适合模型开发者、安全测试人员用于排查训练数据隐性影响。
大模型后训练使用多种不同数据集，每类数据集针对特定行为。但这些数据集除目标模式外还包含附带模式：格式与内容间的关联、跨问题的狭窄表达方式，以及数据筛选过程产生的隐含关联。这些模式对开发者不可见，却对模型显著，导致模型产生出乎意料的行为，例如拒绝以特定题型呈现的真实事实。我们称之为‘数据块式后训练’：模型因不同数据块的独立训练而习得虚假相关性。本文提出SURF——一个运行时黑箱管道，用于暴露此类意外行为；TURF——一种追踪失败源头至具体训练数据的工具。在前导模型（Claude 4.5、GPT-5.1、Grok 4.1、Gemini 3）和开源模型（Tülu 3）上应用，结果显示，数据块式后训练引发校准失准行为，常源于训练数据中不平衡或定义不清的数据块。
原文摘要 · Abstract (English)
LLM post-training involves many diverse datasets, each targeting a specific behavior. But these datasets encode incidental patterns alongside intended ones: correlations between formatting and content, narrow phrasings across diverse problems, and implicit associations arising from the discrete data curation process. These patterns are often invisible to developers yet salient to models, producing behaviors that surprise their creators, such as rejecting true facts presented in a particular question format. We call this chunky post-training: the model learns spurious correlations as a result of distinct chunks of post-training data. We introduce SURF, a black-box pipeline which surfaces these unintended behaviors at run time, and TURF, a tool that traces these failures back to specific post-training data. Applying these tools to frontier models (Claude 4.5, GPT-5.1, Grok 4.1, Gemini 3) and open models (Tülu 3), we show that chunky post-training produces miscalibrated behaviors, which often result from imbalanced or underspecified chunks of post-training data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。