提出通用新颖性评估算法RND，可跨领域自动判断科研创意的创新性。
Enabling AI Scientists to Recognize Innovation: A Domain-Agnostic Algorithm for Assessing Novelty
- 通过比较想法局部密度与邻近密度，实现无领域依赖的新颖性判断。
- 在计算机科学和生物医学领域分别达到0.820和0.765的AUROC，性能领先。
- 跨领域评估中表现稳定，优于现有模型约33%，适合多领域科研自动化。
在追求通用人工智能（AGI）的过程中，自动化生成与评估新研究思路是人工智能驱动科学发现的关键挑战。本文提出相对邻域密度（RND）算法，一种无需领域依赖的新颖性评估方法，通过比较一个研究想法的局部密度与其相邻邻居的密度来克服现有方法的局限性。我们开发了一种可扩展的方法构建测试集，无需专家标注，解决了新颖性评估中的根本难题。利用这些测试集，我们证明RND在计算机科学（AUROC=0.820）和生物医学研究（AUROC=0.765）领域均达到当前最优性能。尤为重要的是，尽管现有模型如Sonnet-3.7及主流度量指标在不同领域间表现下降，RND凭借其领域不变特性，在跨领域评估中保持一致高精度，相较基准模型（0.597）显著提升至0.795。结果验证了RND作为科研新颖性自动化评估的普适解决方案的有效性。
原文摘要 · Abstract (English)
In the pursuit of Artificial General Intelligence (AGI), automating the generation and evaluation of novel research ideas is a key challenge in AI-driven scientific discovery. This paper presents Relative Neighbor Density (RND), a domain-agnostic algorithm for novelty assessment in research ideas that overcomes the limitations of existing approaches by comparing an idea's local density with its adjacent neighbors' densities. We first developed a scalable methodology to create test set without expert labeling, addressing a fundamental challenge in novelty assessment. Using these test sets, we demonstrate that our RND algorithm achieves state-of-the-art (SOTA) performance in computer science (AUROC=0.820) and biomedical research (AUROC=0.765) domains. Most significantly, while SOTA models like Sonnet-3.7 and existing metrics show domain-specific performance degradation, RND maintains consistent accuracies across domains by its domain-invariant property, outperforming all benchmarks by a substantial margin (0.795 v.s. 0.597) on cross-domain evaluation. These results validate RND as a generalizable solution for automated novelty assessment in scientific research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。