用大模型和检索增强，让小模型在低数据下也能高效做形态标注。
Boosting the Capabilities of Compact Models in Low-Data Contexts with Large Language Models and Retrieval-Augmented Generation
- 用语言规则作为输入，通过大模型解析后增强小模型输出
- 在低资源语言上达到新最佳性能，提升显著且更高效
- 适合需要可解释性与高可靠性的语言学研究者
当前语言建模对数据和算力的需求给低资源语言的处理带来挑战。声明性语言知识可通过提供语言特定规则，为模型注入有益归纳偏置，部分缓解数据稀缺问题。本文提出一种基于大语言模型（LLM）的检索增强生成（RAG）框架，用于修正小型模型在形态标注任务中的输出。利用语言学信息弥补数据与参数不足，同时通过LLM解析并提炼书面描述性语法作为输入。实验表明，恰当结合：a) 以语法形式提供的语言学输入，b) LLM的解析能力，以及c) 小型分类网络的可训练性，可实现性能与效率的显著提升。紧凑的RAG支持模型在数据稀缺环境下表现优异，达成该任务及目标语言的新状态水平。本工作还为文献语言学家提供了更可靠、更易用的形态标注工具，可生成有理据的解释与置信度评分。
原文摘要 · Abstract (English)
The data and compute requirements of current language modeling technology pose challenges for the processing and analysis of low-resource languages. Declarative linguistic knowledge has the potential to partially bridge this data scarcity gap by providing models with useful inductive bias in the form of language-specific rules. In this paper, we propose a retrieval augmented generation (RAG) framework backed by a large language model (LLM) to correct the output of a smaller model for the linguistic task of morphological glossing. We leverage linguistic information to make up for the lack of data and trainable parameters, while allowing for inputs from written descriptive grammars interpreted and distilled through an LLM. The results demonstrate that significant leaps in performance and efficiency are possible with the right combination of: a) linguistic inputs in the form of grammars, b) the interpretive power of LLMs, and c) the trainability of smaller token classification networks. We show that a compact, RAG-supported model is highly effective in data-scarce settings, achieving a new state-of-the-art for this task and our target languages. Our work also offers documentary linguists a more reliable and more usable tool for morphological glossing by providing well-reasoned explanations and confidence scores for each output.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。