arXiv:2603.05400cs.CL2026-03中稿 · LREC 2026, 15 page…被引 2

小模型通过推理增强也能精准辨义,省算力还泛化强。

An Exploration-Analysis-Disambiguation Reasoning Framework for Word Sense Disambiguation with Low-Parameter LLMs

  • 用链式思考+邻字分析,让小模型学会逻辑辨义。
  • 40亿参数的Gemma/Qwen在多个数据集上媲美GPT-4-Turbo。
  • 适合资源有限但需高准确率的低算力场景应用。

词义消歧(WSD)是自然语言处理中的关键挑战,尤其在罕见或领域特定词义上常出现误判。尽管大模型如GPT-4-Turbo表现优异,但其计算与能耗限制了可扩展性。本研究探索小于40亿参数的小型开源LLM(如Gemma、Qwen)在经过推理驱动微调后,能否实现相近性能。基于含半自动标注推理过程的FEWS数据集,对八种小型模型进行微调。结果表明,结合链式思考(CoT)与邻字分析的方法,在零样本设置下表现可媲美GPT-4-Turbo。其中,Gemma-3-4B与Qwen-3-4B在FEWS上持续优于所有中等参数基线及现有最优模型,并展现出对未见词义的强泛化能力。在未见过的“Fool Me If You Can”数据集上评估,亦无需任务微调即可跨领域适应。研究表明,通过精心设计的推理导向微调策略,低参数模型可在显著降低计算与能耗的同时,实现高质量的词义消歧。

原文摘要 · Abstract (English)

Word Sense Disambiguation (WSD) remains a key challenge in Natural Language Processing (NLP), especially when dealing with rare or domain-specific senses that are often misinterpreted. While modern high-parameter Large Language Models (LLMs) such as GPT-4-Turbo have shown state-of-the-art WSD performance, their computational and energy demands limit scalability. This study investigates whether low-parameter LLMs (<4B parameters) can achieve comparable results through fine-tuning strategies that emphasize reasoning-driven sense identification. Using the FEWS dataset augmented with semi-automated, rationale-rich annotations, we fine-tune eight small-scale open-source LLMs (e.g. Gemma and Qwen). Our results reveal that Chain-of-Thought (CoT)-based reasoning combined with neighbour-word analysis achieves performance comparable to GPT-4-Turbo in zero-shot settings. Importantly, Gemma-3-4B and Qwen-3-4B models consistently outperform all medium-parameter baselines and state-of-the-art models on FEWS, with robust generalization to unseen senses. Furthermore, evaluation on the unseen "Fool Me If You Can'' dataset confirms strong cross-domain adaptability without task-specific fine-tuning. This work demonstrates that with carefully crafted reasoning-centric fine-tuning, low-parameter LLMs can deliver accurate WSD while substantially reducing computational and energy demands.

词义消歧小模型推理增强低耗能

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。