arXiv:2603.09595cs.CL2026-03

政治学家选NLP模型时,细调通用模型常够用,不必自建专有模型。

Build, Borrow, or Just Fine-Tune? A Political Scientist's Guide to Choosing NLP Models

  • 在冲突事件分类任务中,微调ModernBERT表现接近专用模型
  • 两者差距仅4个百分点,且集中在罕见事件类别中
  • 适合资源有限但需快速落地研究的政治科学家参考

政治科学家在采用自然语言处理工具时面临关键抉择:从头构建领域模型、借用并调整现有模型,还是直接在任务数据上微调通用模型?每种方法在性能、成本和所需专业知识之间权衡不同,但学界缺乏实证指导。本文以冲突事件分类为例,将ModernBERT在全球恐怖主义数据库(GTD)上微调,得到Confli-mBERT,与当前黄金标准的领域专用预训练模型ConfliBERT进行系统比较。Confli-mBERT准确率为75.46%,低于ConfliBERT的79.34%。关键发现是,4个百分点的差距并非均匀分布:在高频攻击类型如爆炸袭击(F1=0.95 vs. 0.96)和绑架(F1=0.92 vs. 0.91)上,两模型几乎无差别;性能差异集中于占总事件不足2%的稀有类别。基于此,本文提出一个实用决策框架,帮助政治科学家判断何时需要专用模型,何时微调通用模型已足够。答案不在于模型本身优劣,而取决于类别频次、容错能力与可用资源的交集。模型、代码与数据已公开于Hugging Face。

原文摘要 · Abstract (English)

Political scientists increasingly face a consequential choice when adopting natural language processing tools: build a domain-specific model from scratch, borrow and adapt an existing one, or simply fine-tune a general-purpose model on task data? Each approach occupies a different point on the spectrum of performance, cost, and required expertise, yet the discipline has offered little empirical guidance on how to navigate this trade-off. This paper provides such guidance. Using conflict event classification as a test case, I fine-tune ModernBERT on the Global Terrorism Database (GTD) to create Confli-mBERT and systematically compare it against ConfliBERT, a domain-specific pretrained model that represents the current gold standard. Confli-mBERT achieves 75.46% accuracy compared to ConfliBERT's 79.34%. Critically, the four-percentage-point gap is not uniform: on high-frequency attack types such as Bombing/Explosion (F1 = 0.95 vs. 0.96) and Kidnapping (F1 = 0.92 vs. 0.91), the models are nearly indistinguishable. Performance differences concentrate in rare event categories comprising fewer than 2% of all incidents. I use these findings to develop a practical decision framework for political scientists considering any NLP-assisted research task: when does the research question demand a specialized model, and when does an accessible fine-tuned alternative suffice? The answer, I argue, depends not on which model is "better" in the abstract, but on the specific intersection of class prevalence, error tolerance, and available resources. The model, training code, and data are publicly available on Hugging Face.

NLP应用模型选择政治科学

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。