研究对话生成中局部修改如何高效传播到全文，提升大模型修稿准确性。
What Else Needs Fixing? Exploring Cost-Effective Test-Time Compute for Revision Propagation in Artifacts Generated Through Conversation
- 通过并行采样与选择机制实现低成本测试时计算
- 准确率提升2.2%至9.7%，最高达93%
- 适合需要高效迭代生成的开发与写作场景
大型语言模型常用于用户通过迭代生成与修订对话来创建文档。当用户仅指定局部修改时，模型需识别相关依赖并将其传播至所有受影响部分。本文研究了在对话生成场景下，大模型对依赖关系传播的能力，并探索该场景下的低成本测试时计算方案。为此，我们构建了一个新基准，评估九种修订方法，涵盖序列反思与并行采样变体，使用 gpt-oss-20b/120b、gpt-5.4-mini 及 qwen3.5-9b/27b/122b 模型。结果表明，基线方法准确率达68.3%–93%，最高效的策略是基于大模型或中位数选择三个并行样本，可提升准确率2.2%–9.7%。代码与数据集已开源。
原文摘要 · Abstract (English)
Large Language Models (LLMs) often help users generate artifacts through iterative cycles of generation and revision in conversation. A challenge here is that, when users specify only a local change during revision, LLMs must instead identify the relevant dependencies and propagate the revision to all affected parts of the artifact. This paper studies this ability of LLMs on conversationally generated artifacts, where the artifact context and its dependencies may be embedded in the conversation history. Toward practical use, we also explore cost-effective test-time compute for this new setting. Specifically, we introduce a new benchmark for this setting, and evaluate nine revision methods, including sequential reflection and parallel sampling variants, using gpt-oss-20b/120b, gpt-5.4-mini, and qwen3.5-9b/27b/122b on the benchmark. The results show that baselines achieve accuracies of 68.3--93%, and the most cost-effective method is selecting from three parallel samples using either LLM-based or medoid selection, which improves accuracy by 2.2--9.7%. Our code and dataset are available at https://github.com/ntt-dkiku/llm-revision-propagation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。