轻量框架实现低资源语言精准词性标注，跨语言迁移仅需三行代码。
FastPOS: Language-Agnostic Scalable POS Tagging Framework Low-Resource Use Case
- 基于Transformer的模块化框架，三行代码即可适配新语言。
- 孟加拉语和印地语词性标注准确率分别达96.85%和97%。
- 适合关注低资源语言NLP的研究者快速部署与优化数据集。
本研究提出一种面向低资源语言的通用型Transformer词性标注框架，以孟加拉语和印地语为案例。仅需三行特定代码即可将模型从孟加拉语迁移到印地语，展现优异可移植性。在两类语言上，词性标注的令牌级准确率分别达到96.85%和97%，即使在数据集不平衡和语言重叠的情况下仍保持高F1值。某特定词性类别表现偏差，反映当前数据集构建仍存挑战。高性能主要归功于底层Transformer结构，通过少量代码调整即可替换。其模块化与开源设计支持快速跨语言适配，降低模型设计与调参负担，使研究者可聚焦于语言预处理与数据集优化，助力非主流语言自然语言处理发展。
原文摘要 · Abstract (English)
This study proposes a language-agnostic transformer-based POS tagging framework designed for low-resource languages, using Bangla and Hindi as case studies. With only three lines of framework-specific code, the model was adapted from Bangla to Hindi, demonstrating effective portability with minimal modification. The framework achieves 96.85 percent and 97 percent token-level accuracy across POS categories in Bangla and Hindi while sustaining strong F1 scores despite dataset imbalance and linguistic overlap. A performance discrepancy in a specific POS category underscores ongoing challenges in dataset curation. The strong results stem from the underlying transformer architecture, which can be replaced with limited code adjustments. Its modular and open-source design enables rapid cross-lingual adaptation while reducing model design and tuning overhead, allowing researchers to focus on linguistic preprocessing and dataset refinement, which are essential for advancing NLP in underrepresented languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。