arXiv:2601.01576cs.IRcs.AI2026-01被引 8

用大模型自动查文献,判断论文创新性是否真实可靠。

OpenNovelty: An LLM-powered Agentic System for Verifiable Scholarly Novelty Assessment

  • 通过提取论文核心任务和贡献生成检索词,精准找相关文献。
  • 对每项贡献进行全文对比,找出被作者忽略的相近工作。
  • 报告带引用和证据片段,可验证、可公开,适合审稿人使用。

评估学术创新性在同行评审中至关重要却极具挑战,因评审者需面对浩繁且快速更新的文献。本文提出 OpenNovelty,一个基于大模型的代理系统,实现透明、基于证据的创新性分析。系统分四步:(1) 提取核心任务与贡献声明以生成检索查询;(2) 利用语义搜索引擎检索相关前期工作;(3) 构建任务相关文献层级分类,并对每一贡献进行全文级比对;(4) 将所有分析整合为带明确引用和证据片段的结构化报告。不同于简单的大模型方法,OpenNovelty 所有判断均基于实际检索到的论文,确保可验证。我们在 500+ 篇 ICLR 2026 提交稿上部署该系统,所有报告已公开。初步分析表明,其能有效识别相关文献,包括作者可能遗漏的紧密相关论文。OpenNovelty 旨在为研究社区提供可扩展工具,推动公平、一致且有证据支持的同行评审。

原文摘要 · Abstract (English)

Evaluating novelty is critical yet challenging in peer review, as reviewers must assess submissions against a vast, rapidly evolving literature. This report presents OpenNovelty, an LLM-powered agentic system for transparent, evidence-based novelty analysis. The system operates through four phases: (1) extracting the core task and contribution claims to generate retrieval queries; (2) retrieving relevant prior work based on extracted queries via semantic search engine; (3) constructing a hierarchical taxonomy of core-task-related work and performing contribution-level full-text comparisons against each contribution; and (4) synthesizing all analyses into a structured novelty report with explicit citations and evidence snippets. Unlike naive LLM-based approaches, \textsc{OpenNovelty} grounds all assessments in retrieved real papers, ensuring verifiable judgments. We deploy our system on 500+ ICLR 2026 submissions with all reports publicly available on our website, and preliminary analysis suggests it can identify relevant prior work, including closely related papers that authors may overlook. OpenNovelty aims to empower the research community with a scalable tool that promotes fair, consistent, and evidence-backed peer review.

学术评审大模型应用创新性评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。