arXiv:2606.32002cs.AIcs.LG2026-06

自生成问答训练存在隐蔽缺陷,易被文本格式干扰并产生误导性答案。

Self-Study Reconsidered: The Hidden Fragility of Learning from Self-Generated QA

论文配图:Self-Study Reconsidered: The Hidden Fragility of Learning from Self-Generated QA
图 1 · 摘自论文原文
  • 通过固定问题对应目标,减少提问时的偏见选择
  • 过滤指令类片段后,答案错误率从88%降至13%
  • 适合关注模型训练数据质量的研究者

语言模型越来越多地采用自生成问答(QA)进行训练:模型从文档中生成问题,并用同一文本回答,形成监督信号用于微调或知识压缩。我们发现,这一生成过程并非中立预处理,而是隐含策略,既决定哪些证据成为训练信号,也影响其回答方式,且在两个阶段均存在脆弱性。在提问阶段,生成器不均匀扫描文档,覆盖早期饱和,集中于显著片段;不同提示趋向相同区域,问题选择受局部呈现方式驱动。因此,如未清理的标记符号等显著噪声可能劫持跨模型、跨规模的提问。在回答阶段,生成模型倾向于服从文本中嵌入的指令式段落,这种服从性取决于语义意图和表面形式而非严格程度,在任务冲突下尤为严重,且大模型更易服从。这些失败模式源于生成环节的选择,无需改变训练流程即可缓解:将每个问题绑定固定目标可减少偏见选择;预先过滤指令类段落后,平均注入服从率从88%降至13%,同时保留几乎所有干净文本。

原文摘要 · Abstract (English)

Language models are increasingly taught from synthetic question--answer (QA) supervision: a model generates questions about a document, answers them from the same text, and the resulting pairs are used to fine-tune, distill, or compress knowledge into another model. We show that this generation step is not neutral preprocessing. It is an implicit policy that both selects which evidence becomes training signal and decides how that evidence is answered, and it is fragile at both stages. When choosing what to ask, generators do not scan a document uniformly. Coverage saturates early and concentrates on salient spans, diverse prompts converge on the same regions, and what looks question-worthy is driven by local presentation. As a result, salient artifacts such as poorly cleaned markup can hijack question generation across model families and scales. When answering, the model that produces the supervision tends to obey instruction-like passages embedded in the text. This compliance depends on the intent and surface form of the passage rather than its strictness, and is worst under task conflict, where larger models comply more often. These failure modes arise from choices made during QA generation, so they can be reduced without changing the training loop. Tying each question to a fixed target reduces biased selection, and filtering instruction-like spans before answering lowers mean injection compliance from $88\%$ to $13\%$ in our evaluation while retaining nearly all clean text.

自生成问答模型偏差训练数据语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。