用音素表征解决韩语未登录词问题，提升模型对生僻词的处理能力。
Handling Korean Out-of-Vocabulary Words with Phoneme Representation Learning
- 利用韩语音素与字母高度相关特性，融合音素与词表征
- 在多个韩语NLP任务上平均提升1.9%，优于现有最优模型
- 可即插即用，适配主流静态与上下文嵌入模型
本文提出KOPL框架，用于解决韩语未登录词（OOV）问题。基于韩语作为音素文字的特性，其音素与字符高度相关，KOPL结合音素与词表征，使韩语未登录词能同时捕捉文本和音素信息。实验证明，该方法显著提升韩语自然语言处理任务性能，且可无缝集成至现有静态与上下文嵌入模型中，实现即插即用。特别地，相比当前最优模型，平均提升1.9%。代码已公开于https://github.com/jej127/KOPL.git。
原文摘要 · Abstract (English)
In this study, we introduce KOPL, a novel framework for handling Korean OOV words with Phoneme representation Learning. Our work is based on the linguistic property of Korean as a phonemic script, the high correlation between phonemes and letters. KOPL incorporates phoneme and word representations for Korean OOV words, facilitating Korean OOV word representations to capture both text and phoneme information of words. We empirically demonstrate that KOPL significantly improves the performance on Korean Natural Language Processing (NLP) tasks, while being readily integrated into existing static and contextual Korean embedding models in a plug-and-play manner. Notably, we show that KOPL outperforms the state-of-the-art model by an average of 1.9%. Our code is available at https://github.com/jej127/KOPL.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。