用大模型提升拼音输入法的准确率，让残障用户更高效交流
Adapting Large Language Models for Character-based Augmentative and Alternative Communication
- 将子词级大模型转换为字符预测，提升输入准确性
- 在对话文本上比基线方法错误率降低18.7%
- 专为辅助沟通场景优化，适合残障人士语音输入系统
使用辅助与替代沟通（AAC）的用户可通过逐字符输入界面进行交流。然而，当前主流的大规模预训练语言模型通常预测可变长度的子词单元。本文研究如何实际利用此类模型实现高精度、高效率的字符预测。提出的算法通过子词大模型生成字符预测，相比分类层、字节级模型或n-gram模型表现更优。此外，我们基于对句子在口语和书面语中沟通效用的评分，构建了一个大规模语料库，并在此基础上设计了一种领域适配方法。实验表明，该方法进一步提升了模型在简单对话文本上的性能。
原文摘要 · Abstract (English)
Users of Augmentative and Alternative Communication (AAC) may write letter-by-letter via an interface that uses a character language model. However, most state-of-the-art large pretrained language models predict subword tokens of variable length. We investigate how to practically use such models to make accurate and efficient character predictions. Our algorithm for producing character predictions from a subword large language model (LLM) provides more accurate predictions than using a classification layer, a byte-level LLM, or an n-gram model. Additionally, we investigate a domain adaptation procedure based on a large dataset of sentences we curated based on scoring how useful each sentence might be for spoken or written AAC communication. We find our procedure further improves model performance on simple, conversational text.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。