arXiv:2601.12389cs.CLcs.AI2026-01AAAI

提出NADIR模型,实现印地语翻译的快速高准度转换

NADIR: Differential Attention Flow for Non-Autoregressive Transliteration in Indic Languages

  • 采用差分注意力流与专家混合机制,摆脱序列依赖
  • 推理速度提升13倍,字符错误率仅比自回归模型高1.34%
  • 显著降低重复、替换、遗漏、插入错误,适合实时部署

本文指出,并非所有序列到序列任务都需自回归(AR)模型强归纳偏置。多语言转写、代码重构、语法纠错或文本规范化等任务通常依赖局部依赖,而全序列建模能力对这些任务而言是冗余的,导致准确率与推理延迟之间的权衡。非自回归(NAR)模型虽速度快,但常出现幻觉和长度控制差的问题。为此,本文聚焦印地语多语言转写任务,提出NADIR——一种新型NAR架构,在速度与准确率间取得平衡。NADIR融合差分变压器与专家混合机制,无需序列依赖即可稳健建模复杂字符映射。相比最先进AR基线,其推理速度提升超过13倍;在平均字符错误率上达到15.78%,仅略高于AR模型的14.44%,优于标准NAR模型的21.88%。更重要的是,NADIR将重复错误降低49.53%,替换错误降低24.45%,遗漏错误降低32.92%,插入错误降低16.87%。该工作为构建快速可靠的NAR系统提供了实用蓝图,有效弥合了AR精度与实时大规模部署需求之间的差距。

原文摘要 · Abstract (English)

In this work, we argue that not all sequence-to-sequence tasks require the strong inductive biases of autoregressive (AR) models. Tasks like multilingual transliteration, code refactoring, grammatical correction or text normalization often rely on local dependencies where the full modeling capacity of AR models can be overkill, creating a trade-off between their high accuracy and high inference latency. While non-autoregressive (NAR) models offer speed, they typically suffer from hallucinations and poor length control. To explore this trade-off, we focus on the multilingual transliteration task in Indic languages and introduce NADIR, a novel NAR architecture designed to strike a balance between speed and accuracy. NADIR integrates a Differential Transformer and a Mixture-of-Experts mechanism, enabling it to robustly model complex character mappings without sequential dependencies. NADIR achieves over a 13x speed-up compared to the state-of-the-art AR baseline. It maintains a competitive mean Character Error Rate of 15.78%, compared to 14.44% for the AR model and 21.88% for a standard NAR equivalent. Importantly, NADIR reduces Repetition errors by 49.53%, Substitution errors by 24.45%, Omission errors by 32.92%, and Insertion errors by 16.87%. This work provides a practical blueprint for building fast and reliable NAR systems, effectively bridging the gap between AR accuracy and the demands of real-time, large-scale deployment.

非自回归转写差分注意力印地语

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。