用AI代理框架自动评审天文望远镜申请,提升效率与透明度。
AstroReview: An LLM-driven Multi-Agent Framework for Telescope Proposal Peer Review and Refinement
- 分三阶段自动化评审:科学价值、可行性、可靠性验证
- 无需微调,第三阶段准确率达87%,识别真通过提案
- 迭代反馈使修改稿通过率提升66%,适合资源受限望远镜
现代天文台观测时间竞争加剧,申请量超过可用时间,及时、一致且透明的同行评审成为天文学发展的关键瓶颈。自动化该流程在科学和操作上都至关重要,可实现公平分配与可复现决策。我们提出AstroReview,一个开源的基于代理的框架,分三个阶段自动化申请评审:(i) 创新性与科学价值,(ii) 可行性与预期产出,(iii) 元评审与可靠性验证。任务隔离与显式推理路径有效减少幻觉,提升透明度。实验中,仅使用最后一阶段的AstroReview在无领域微调情况下,正确识别真实通过提案的准确率达到87%。AstroReview in Action模块模拟评审与优化循环;结合提案撰写代理,经两次迭代后修订稿的通过率提升66%,表明迭代反馈结合自动化元评审与可靠性验证能显著提升质量。这些结果为资源有限设施提供了可扩展、可审计、高吞吐量评审的可行路径。
原文摘要 · Abstract (English)
Competitive access to modern observatories has intensified as proposal volumes outpace available telescope time, making timely, consistent, and transparent peer review a critical bottleneck for the advancement of astronomy. Automating parts of this process is therefore both scientifically significant and operationally necessary to ensure fair allocation and reproducible decisions at scale. We present AstroReview, an open-source, agent-based framework that automates proposal review in three stages: (i) novelty and scientific merit, (ii) feasibility and expected yield, and (iii) meta-review and reliability verification. Task isolation and explicit reasoning traces curb hallucinations and improve transparency. Without any domain specific fine tuning, AstroReview used in our experiments only for the last stage, correctly identifies genuinely accepted proposals with an accuracy of 87%. The AstroReview in Action module replicates the review and refinement loop; with its integrated Proposal Authoring Agent, the acceptance rate of revised drafts increases by 66% after two iterations, showing that iterative feedback combined with automated meta-review and reliability verification delivers measurable quality gains. Together, these results point to a practical path toward scalable, auditable, and higher throughput proposal review for resource limited facilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。