小模型通过任务适配，能在通用和文学关系抽取上超越大模型。
Sub-Billion, Super-Frontier: Small Language Models Rival Zero-Shot Frontier LLMs on General and Literary Relation Extraction
- 用小模型在特定任务数据上微调，替代大模型零样本推理。
- 0.5B参数模型在通用文本上达到0.83的F1，超过GPT-5.4和Claude。
- 适合资源受限、注重隐私的部署场景，可跑在消费级显卡上。
大型语言模型（LLMs）在关系抽取（RE）上表现优异，但其计算开销和对专有API的依赖限制了在资源受限或隐私敏感环境中的部署。本文研究小型语言模型（SLMs）在通用领域和文学文本中能否缩小这一差距。评估了5个360M至3B参数的模型，在三种领域构成和两种提示微调方式下共30种配置，对比零样本前沿大模型与判别式RoBERTa基线。在九个基准上，最佳子十亿参数模型Qwen2.5-0.5B在混合通用数据上微调后，通用领域正类微F1达0.83，优于零样本的GPT-5.4（0.69）和Claude Sonnet 4.6（0.66）。这并非因小模型更强大，而是针对性任务适配使4比特模型可在单张消费级显卡上部署并超越通用大模型。同域RoBERTa基线也超过两者，表明收益来自任务适配而非生成解码。在文学关系抽取上，微调后的小模型在人工标注的传记基准上达0.92，优于GPT-5.4的0.83；在两个文学基准平均上分别为0.833与0.578。定向领域适应预训练未带来实际提升，家族内缩放比较也仅边际改善。结果表明，当有特定任务数据时，紧凑的任务适配模型可实现准确、私密且硬件高效的抽取。
原文摘要 · Abstract (English)
Large language models (LLMs) achieve strong relation extraction (RE), but their computational demands and reliance on proprietary APIs limit deployment in resource-constrained or privacy-sensitive settings. We investigate how far small language models (SLMs) can close this gap across general-domain and literary text. We evaluate five models from 360M to 3B parameters under three domain-composition regimes and two prompt-conditioned tuning styles (30 configurations), comparing them with zero-shot frontier LLMs and a discriminative RoBERTa baseline. Across nine benchmarks, the best sub-billion model, Qwen2.5-0.5B fine-tuned on pooled general-domain data, achieves a general-domain positive-class micro-F1 of 0.83, versus 0.69 for GPT-5.4 and 0.66 for Claude Sonnet 4.6 evaluated zero-shot. This does not imply that SLMs are intrinsically stronger; rather, targeted task adaptation enables 4-bit models deployable on a single consumer GPU to outperform general-purpose frontier systems under this protocol. An in-domain RoBERTa baseline also exceeds both frontier models, indicating that the gain stems from task adaptation rather than generative decoding. On literary RE, tuned SLMs reach 0.92 on the human-annotated Biographical benchmark versus 0.83 for GPT-5.4, and 0.833 versus 0.578 on the two-benchmark literary average. A targeted domain-adaptive pretraining case study yields no practically meaningful gain over supervised fine-tuning, while the cleanest within-family scale comparison shows only marginal improvement. These results show that, when task-specific data are available, compact task-adapted models can provide accurate, private, and hardware-efficient RE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。