自动识别NLP研究背后的实用目标与科学动机，揭示研究真实意图。
Social Good or Scientific Curiosity? Uncovering the Research Framing Behind NLP Artefacts
- 构建三模块系统，从论文中提取目标、手段和利益相关方并建立关联。
- 在事实核查与仇恨言论检测任务上优于强基线模型，准确率提升显著。
- 发现近年研究趋向模糊目标、重科学探索轻实际应用，倾向辅助人工而非完全自动化。
明确自然语言处理成果（如模型、数据集）的研究框架，对推动研究与实际应用对齐至关重要。近期研究手动分析显示，多数论文未清晰说明关键利益相关方、预期用途或适用情境。本文提出自动化分析方法，构建三组件系统：首先提取关键要素（手段、目标、利益相关方），再通过可解释规则与上下文推理建立关联。我们在两个领域评估：使用现有数据集进行自动事实核查，以及为仇恨言论检测新标注数据集——均取得优于强基线大模型的稳定提升。最后，将该系统应用于近期自动事实核查论文，发现三个趋势：研究目标日益模糊或不明确，科学探索比重上升而应用导向减弱，研究重心转向支持人工核查者而非追求完全自动化。
原文摘要 · Abstract (English)
Clarifying the research framing of NLP artefacts (e.g., models, datasets, etc.) is crucial to aligning research with practical applications. Recent studies manually analyzed NLP research across domains, showing that few papers explicitly identify key stakeholders, intended uses, or appropriate contexts. In this work, we propose to automate this analysis, developing a three-component system that infers research framings by first extracting key elements (means, ends, stakeholders), then linking them through interpretable rules and contextual reasoning. We evaluate our approach on two domains: automated fact-checking using an existing dataset, and hate speech detection for which we annotate a new dataset-achieving consistent improvements over strong LLM baselines. Finally, we apply our system to recent automated fact-checking papers and uncover three notable trends: a rise in vague or underspecified research goals, increased emphasis on scientific exploration over application, and a shift toward supporting human fact-checkers rather than pursuing full automation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。