用AI检测论文回复中的承诺履行情况，提升审稿透明度。
Commitment Checklist: Auditing Author Commitments in Peer Review
- 用大模型比对审稿回复与最终论文，自动识别承诺项
- 25%承诺未兑现，实验缺失等关键项最常见
- 提出承诺清单机制，适合关注审稿公正的学者
同行评审作者回复中常包含补充实验、开源代码或澄清内容等承诺，但目前缺乏系统性机制确保兑现。本文通过大语言模型（LLMs）对ICLR-2025和EMNLP-2024的审稿回复与最终版论文进行大规模审计，发现多数承诺得以实施，但约25%未完成，尤其是‘缺失实验’等高影响力项。研究证明，基于LLM的工具可有效识别承诺项。我们提出‘作者承诺清单’概念，可提醒作者与组织者未完成的承诺，增强责任意识，提升审稿流程的完整性。文中讨论该实践的优势，并呼吁未来会议采纳。
原文摘要 · Abstract (English)
Peer review author responses often include commitments to add experiments, release code, or clarify content in the final paper. Yet, there is currently no systematic mechanism to ensure authors fulfill these promises. In this position paper, we present a large-scale audit of author commitments using large language models (LLMs) to compare rebuttals against camera-ready versions. Analyzing the commitments from ICLR-2025 and EMNLP-2024, we find that while a majority of promised changes are implemented, a significant share (about 25%) are not, with "missing experiments" and other high-impact items among the most frequently unfulfilled. We demonstrate that LLM-based tools can feasibly detect the promises. Finally, we propose the idea of Author Commitment Checklist, which would alert authors and organizers to unaddressed promises, increasing accountability and strengthening the integrity of the peer review process. We discuss the benefits of this practice and advocate for its adoption in future conferences.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。