评测大模型在人文社科复杂引用场景下的提取与解析能力，发现小模型微调+分步处理更有效。
Benchmarking Large Language Models on Reference Extraction and Parsing in the Social Sciences and Humanities
- 构建三套真实人文社科引用数据集，覆盖多语言、脚注嵌入等复杂场景。
- 大模型在引用提取上已达饱和，但解析和端到端处理仍存在明显瓶颈。
- 轻量级微调和分步处理可显著提升鲁棒性，适合混合部署策略。
参考文献提取与解析是引文索引、链接及学术知识图谱构建的基础。然而，现有评估大多聚焦于干净的英文末尾参考文献列表，难以代表人文社科（SSH）领域——该领域引用常为多语言、嵌入脚注、缩写且受多样历史惯例影响。本文提出一个统一基准，涵盖三个互补数据集：CEX（跨学科英文期刊文章）、EXCITE（德英双语文档，含段末、仅脚注及混合模式）和LinkedBooks（人文类参考文献，风格差异大、多语言）。在结构化约束下，评估三个难度递增任务：参考文献提取、解析与端到端文档解析。对比强监督基线（GROBID）与当代大模型（DeepSeek-V3.1、Mistral-Small-3.2-24B、Gemma-3-27B-it、Qwen3-VL（4B-32B变体））。结果显示，提取性能在中等能力阈值后趋于饱和，而解析与端到端解析仍是主要瓶颈，源于对噪声排版的结构输出脆弱性。轻量级LoRA微调带来持续增益，尤其在重SSH数据集上；分段与流水线处理显著提升鲁棒性。最后主张采用路由式混合部署：对结构清晰、分布内PDF使用GROBID，对多语言及脚注密集文档则交由任务适配的大模型处理。
原文摘要 · Abstract (English)
Bibliographic reference extraction and parsing are foundational for citation indexing, linking, and downstream scholarly knowledge-graph construction. However, most established evaluations focus on clean, English, end-of-document bibliographies, and therefore underrepresent the Social Sciences and Humanities (SSH), where citations are frequently multilingual, embedded in footnotes, abbreviated, and shaped by heterogeneous historical conventions. We present a unified benchmark that targets these SSH-realistic conditions across three complementary datasets: CEX (English journal articles spanning multiple disciplines), EXCITE (German/English documents with end-section, footnote-only, and mixed regimes), and LinkedBooks (humanities references with strong stylistic variation and multilinguality). We evaluate three tasks of increasing difficulty -- reference extraction, reference parsing, and end-to-end document parsing -- under a schema-constrained setup that enables direct comparison between a strong supervised pipeline baseline (GROBID) and contemporary LLMs (DeepSeek-V3.1, Mistral-Small-3.2-24B, Gemma-3-27B-it, and Qwen3-VL (4B-32B variants)). Across datasets, extraction largely saturates beyond a moderate capability threshold, while parsing and end-to-end parsing remain the primary bottlenecks due to structured-output brittleness under noisy layouts. We further show that lightweight LoRA adaptation yields consistent gains -- especially on SSH-heavy benchmarks -- and that segmentation/pipelining can substantially improve robustness. Finally, we argue for hybrid deployment via routing: leveraging GROBID for well-structured, in-distribution PDFs while escalating multilingual and footnote-heavy documents to task-adapted LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。