用随机采样微调编码器，比开源GPT更准地提取天文学知识
Encoder Fine-tuning with Stochastic Sampling Outperforms Open-weight GPT in Astronomy Knowledge Extraction
- 基于SciBERT的多任务模型，随机采样训练片段提升泛化能力
- 在望远镜、仪器等实体识别上超越开源GPT基线，性能显著提升
- 适合需要低成本高精度知识提取的天文文献自动化场景
天文学科研文献快速增长，自动化提取关键实体与上下文信息愈发重要。本文提出一种基于编码器的系统，用于从天文论文中提取知识。目标包括分类望远镜引用、检测辅助语义属性、识别仪器提及。我们构建了一个基于SciBERT的多任务Transformer模型，并在天文语料上进行微调。微调时采用随机采样训练片段，推理时对测试片段进行多数投票。该系统虽简单且成本低，但显著优于开放权重的GPT基线。
原文摘要 · Abstract (English)
Scientific literature in astronomy is rapidly expanding, making it increasingly important to automate the extraction of key entities and contextual information from research papers. In this paper, we present an encoder-based system for extracting knowledge from astronomy articles. Our objective is to develop models capable of classifying telescope references, detecting auxiliary semantic attributes, and recognizing instrument mentions from textual content. To this end, we implement a multi-task transformer-based system built upon the SciBERT model and fine-tuned for astronomy corpora classification. To carry out the fine-tuning, we stochastically sample segments from the training data and use majority voting over the test segments at inference time. Our system, despite its simplicity and low-cost implementation, significantly outperforms the open-weight GPT baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。