用检索增强方法评估AI论文概念新颖性，区分真创新与增量改进。
NoveltyRank: A Retrieval-Augmented Framework for Conceptual Novelty Estimation in AI Research
- 结合语义表示与文献检索，从二分类和排序双角度评估新颖性。
- 轻量级微调模型表现优于大模型，证明任务监督比参数规模更重要。
- 已上线实时评分系统，支持公众交互与论文新颖性检测。
科学论文发表速度加快，难以在增量研究中识别真正原创的工作。我们提出一种框架，通过结合语义表示学习与对先前文献的检索式比较，估计研究论文的概念新颖性。将新颖性建模为二分类任务（新颖 vs. 非新颖）和成对排序任务（相对新颖性），实现绝对与相对评估。实验对比了三种模型规模，涵盖小型领域专用编码器到零样本前沿模型。结果显示，经过微调的轻量级模型虽参数更少，但性能优于更大的零样本模型，表明任务特定监督对概念新颖性估计的重要性超过模型规模。我们进一步将表现最佳的模型部署为在线系统，支持公众交互与实时新颖性评分。
原文摘要 · Abstract (English)
The accelerating pace of scientific publication makes it difficult to identify truly original research among incremental work. We propose a framework for estimating the conceptual novelty of research papers by combining semantic representation learning with retrieval-based comparison against prior literature. We model novelty as both a binary classification task (novel vs. non-novel) and a pairwise ranking task (comparative novelty), enabling absolute and relative assessments. Experiments benchmark three model scales, ranging from compact domain-specific encoders to a zero-shot frontier model. Results show that fine-tuned lightweight models outperform larger zero-shot models despite their smaller parameter count, indicating that task-specific supervision matters more than scale for conceptual novelty estimation. We further deploy the best-performing model as an online system for public interaction and real-time novelty scoring.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。