混合小模型与大模型，提升中文错别字纠正效果
Mixture of Small and Large Models for Chinese Spelling Check
- 动态融合小模型精准修正与大模型语言流畅性
- 在多个数据集上达到当前最优性能
- 无需微调大模型，适合快速适配新领域
大型语言模型（LLM）在中文错别字检查（CSC）任务中虽有广泛应用，但表现仍不理想。相比之下，基于微调的BERT模型依赖高质量领域数据，表现优异但易过拟合编辑模式。本文提出一种动态混合方法，在束搜索解码阶段融合小模型与大模型的概率分布，实现小模型的精确修正与大模型的语言流畅性平衡。该方法无需微调大模型，显著节省时间与资源，有利于领域迁移。大量实验表明，该混合方法显著提升纠错能力，在多个数据集上取得当前最优结果。代码已公开于 https://github.com/zhqiao-nlp/MSLLM。
原文摘要 · Abstract (English)
In the era of large language models (LLMs), the Chinese Spelling Check (CSC) task has seen various LLM methods developed, yet their performance remains unsatisfactory. In contrast, fine-tuned BERT-based models, relying on high-quality in-domain data, show excellent performance but suffer from edit pattern overfitting. This paper proposes a novel dynamic mixture approach that effectively combines the probability distributions of small models and LLMs during the beam search decoding phase, achieving a balanced enhancement of precise corrections from small models and the fluency of LLMs. This approach also eliminates the need for fine-tuning LLMs, saving significant time and resources, and facilitating domain adaptation. Comprehensive experiments demonstrate that our mixture approach significantly boosts error correction capabilities, achieving state-of-the-art results across multiple datasets. Our code is available at https://github.com/zhqiao-nlp/MSLLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。