arXiv:2606.30914cs.CL2026-06

评测孟加拉语事件检测模型在噪声文本中的表现,发现解码器模型更抗噪。

Beyond Clean Text: Evaluating Encoder and Decoder Robustness for Bangla Event Detection in Noisy Text

论文配图:Beyond Clean Text: Evaluating Encoder and Decoder Robustness for Bangla Event Detection in Noisy Text
图 1 · 摘自论文原文
  • 对比编码器与解码器架构在噪声下的事件检测能力
  • 解码器模型在触发词受损时仍保持较高准确率
  • 联合训练可显著提升编码器模型的鲁棒性

事件检测系统通常在干净文本上评估,对其在真实噪声环境下的鲁棒性研究不足,尤其针对低资源语言如孟加拉语。本文构建了一个涵盖40种事件子类型的孟加拉语新闻事件本体,并建立基准数据集,包含9,979条标注句,覆盖干净新闻文本、实际语音识别(ASR)转录和拼写错误文本。系统评估了微调的编码器模型(BanglaBERT、XLM-R)与指令微调的解码器模型(Llama 3、Gemma 3)。结果表明:编码器模型在干净文本上表现更优,但在噪声下性能大幅下降;而解码器大模型显著更鲁棒,尤其在事件触发词被破坏时。进一步发现,在指令微调中引入标注指南可提升噪声文本基线性能,但对不同噪声条件的降级缓解效果不一致。模型规模扩大持续增强解码器模型的鲁棒性,而同时在干净与噪声数据上联合训练是一种有效正则化策略,能显著缩小编码器模型的鲁棒性差距。

原文摘要 · Abstract (English)

Event detection (ED) systems are typically evaluated on clean, curated text, leaving their robustness to real-world noise largely unexplored, particularly for low-resource languages such as Bangla. We introduce a generalized Bangla news event ontology and a benchmark comprising 9,979 annotated sentences across 40 event subtypes, spanning clean news text, real-world Automatic Speech Recognition (ASR) transcripts, and orthographically corrupted text. We systematically evaluate fine-tuned encoder-only models (BanglaBERT and XLM-R) alongside instruction-tuned decoder-only large language models (Llama 3 and Gemma 3). Our results reveal a clear architectural trade-off: encoder models achieve higher performance on clean text but degrade substantially under noise, whereas decoder-only LLMs are markedly more robust, particularly when event triggers are corrupted. We further show that embedding annotation guidelines during instruction tuning establishes a higher performance baseline on noisy text but yields inconsistent reductions in performance degradation across noisy conditions. Finally, model scaling consistently improves the robustness of decoder-only LLMs, while combined training on clean and noisy data serves as an effective regularization strategy that disproportionately benefits encoder architectures, significantly narrowing the robustness gap.

事件检测低资源语言鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。