区分操作数与操作符扰动，发现操作符多样性比精确度更重要
Quantifying Error Tolerance in Synthetic Data: An Atomic-level Operand vs. Operator Perturbation Study
- 将数据分解为函数单元，区分可容忍的操作数扰动与致命的操作符扰动
- 模型对操作数扰动鲁棒，但对操作符扰动极敏感，导致性能骤降
- 优先保证操作符多样性可提升模型性能，优于传统严格过滤策略
合成数据生成已成为推动大语言模型发展的核心手段。然而，缺乏对错误容错能力的定量分析，成为当前瓶颈。现有筛选策略在极端之间摇摆：过于激进会误删有价值样本，过于宽松则无法有效剔除错误样本。为此，本文提出原子树操作建模（ATOM）框架，将数据分解为功能单元（f(x)→y），区分良性操作数x扰动与致命操作符f扰动。前者常被激进过滤无谓丢弃，后者则易被宽松过滤遗漏。实验揭示双重分离现象：模型对操作数扰动具有鲁棒性，但面对操作符扰动时性能崩溃。通过优先保障操作符而非过度追求操作数精度，基于ATOM生成的数据超越严格基线（如相比LIMA提升+3.1%），表明操作符多样性比操作数精度更为关键。代码已开源：https://github.com/Lut-hub/ATOM。
原文摘要 · Abstract (English)
Synthetic data generation has become a cornerstone for advancing large language models. However, the lack of the quantitative analysis for error tolerance became a critical bottleneck. Consequently, current filtering strategies fluctuate between two extremes: they are either overly aggressive, risking the exclusion of potentially valuable samples, or overly permissive, failing to eliminate erroneous samples effectively. To bridge this gap, this paper introduces Atomic Tree Operation Modeling (ATOM), a framework that decomposes data into functional units ($f(x)\rightarrow y$). ATOM distinguishes benign Operand $x$ perturbations from fatal Operator $f$ perturbations. The former are needlessly discarded by aggressive filtering, while the latter slip through permissive filtering. Our experiments reveal a double dissociation: models are robust to operand perturbations but collapse under operator perturbations. By prioritizing operator over aggressive operand precision, our ATOM-synthesized data outperforms rigorous baselines (e.g., +3.1% gain over LIMA), suggesting that operator diversity matters more than operand precision. Our code is available at https://github.com/Lut-hub/ATOM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。