13位天文学家实测大模型，发现它能帮写论文但需人工把关。
What is the Role of Large Language Models in the Evolution of Astronomy Research?
- 邀请13位天文学家长期试用大模型完成研究任务
- 模型在文献综述、写作等环节效率高但有事实错误
- 适合需要快速构思的科研人员，需警惕幻觉风险
我们对13位处于不同职业阶段和研究领域的天文学家进行了为期数月的研究，考察大语言模型（LLMs）在各类科研任务中的应用表现，并辅以匿名问卷评估其使用体验与态度。研究涵盖了创意构思、文献综述、代码编写、论文撰写及科普传播等任务，提供了具体输出示例。结果表明，尽管这些模型在提升工作效率方面具有潜力，但仍存在事实性错误、逻辑不严谨等问题。我们强调，研究人员必须结合批判性思维与领域专业知识，将LLMs作为辅助工具而非替代品，以保障科学严谨性。同时讨论了通用及科研场景下的伦理问题，并提出多项建议。
原文摘要 · Abstract (English)
ChatGPT and other state-of-the-art large language models (LLMs) are rapidly transforming multiple fields, offering powerful tools for a wide range of applications. These models, commonly trained on vast datasets, exhibit human-like text generation capabilities, making them useful for research tasks such as ideation, literature review, coding, drafting, and outreach. We conducted a study involving 13 astronomers at different career stages and research fields to explore LLM applications across diverse tasks over several months and to evaluate their performance in research-related activities. This work was accompanied by an anonymous survey assessing participants' experiences and attitudes towards LLMs. We provide a detailed analysis of the tasks attempted and the survey answers, along with specific output examples. Our findings highlight both the potential and limitations of LLMs in supporting research while also addressing general and research-specific ethical considerations. We conclude with a series of recommendations, emphasizing the need for researchers to complement LLMs with critical thinking and domain expertise, ensuring these tools serve as aids rather than substitutes for rigorous scientific inquiry.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。