大模型在反复迭代中常违背原始约束,却能准确复述这些约束。
Models Recall What They Violate: Constraint Adherence in Multi-Turn LLM Ideation

- 构建多轮交互基准DriftBench,评估大模型在科研创意中的约束遵守情况。
- 8%至99%的模型存在‘知道但违反’现象,复杂度随迭代显著上升。
- 人类验证表明模型评判保守,适合研究人机协作与设计一致性问题。
当研究人员与大语言模型进行多轮迭代以优化创意时,模型是否保持对初始目标的忠实?我们提出DriftBench,一个用于评估多轮大模型辅助科学创意中约束遵守情况的基准。在涵盖2,146次评分运行、七种模型(来自五家厂商,含两个开源模型)、四种交互条件和38份跨24个科学领域的研究简报的测试中,发现迭代压力会显著增加结构复杂性,且通常降低对原始约束的遵守程度。重述探测显示,模型在行为上违背约束的同时仍能准确复述这些约束,存在‘知道但违反’(KBV)现象,其比率在不同模型间为8%至99%。结构化检查点部分缓解了KBV率,但未能消除这种分离,且复杂度膨胀持续存在。人类盲评验证表明,模型裁判低估了约束违规,因此报告的遵守分数偏保守。敏感性分析确认结果在温度(0.7 vs. 1.0)和压力类型(新颖性 vs. 严谨性)下均稳健。所有研究简报、提示、评分标准、对话记录和评分数据均已开源。
原文摘要 · Abstract (English)
When researchers iteratively refine ideas with large language models, do the models preserve fidelity to the original objective? We introduce DriftBench, a benchmark for evaluating constraint adherence in multi-turn LLM-assisted scientific ideation. Across 2,146 scored benchmark runs spanning seven models from five providers (including two open-weight), four interaction conditions, and 38 research briefs from 24 scientific domains, we find that iterative pressure reliably increases structural complexity and often reduces adherence to original constraints. A restatement probe reveals a dissociation between declarative recall and behavioral adherence, as models accurately restate constraints they simultaneously violate. The knows-but-violates (KBV) rate, measuring constraint non-compliance despite preserved recall, ranges from 8% to 99% across models. Structured checkpointing partially reduces KBV rates but does not close the dissociation, and complexity inflation persists. Human validation against blind raters confirms that the LLM judge under-detects constraint violations, making reported constraint adherence scores conservative. Sensitivity analyses confirm the findings are robust to temperature (0.7 vs.\ 1.0) and pressure type (novelty vs.\ rigor). We release all briefs, prompts, rubrics, transcripts, and scores as an open benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。