用图像表示单词，提升语言模型抗乱码攻击能力
Enhancing Robustness of Autoregressive Language Models against Orthographic Attacks via Pixel-based Approach
- 将词语转为图像，避免字符乱码导致的词汇缺失
- 在多语言数据集上表现更稳定，错误率显著降低
- 适合需要高鲁棒性的跨语言应用开发
自回归语言模型易受拼写攻击影响，即通过引入多语言字符对输入文本进行扰动，导致性能大幅下降。这一漏洞主要源于子词分词器及其嵌入对未登录词的处理缺陷。为此，我们提出一种基于像素的生成式语言模型，将文本嵌入替换为图像表示，通过将每个词渲染为独立图像实现更强的抗噪能力，并支持跨多种书写系统的多语言文本兼容。我们在多语言LAMBADA数据集、WMT24数据集和SST-2基准上评估该方法，验证了其在抵御拼写噪声方面的鲁棒性及在多语言场景下的有效性。
原文摘要 · Abstract (English)
Autoregressive language models are vulnerable to orthographic attacks, where input text is perturbed with characters from multilingual alphabets, leading to substantial performance degradation. This vulnerability primarily stems from the out-of-vocabulary issue inherent in subword tokenizers and their embeddings. To address this limitation, we propose a pixel-based generative language model that replaces the text-based embeddings with pixel-based representations by rendering words as individual images. This design provides stronger robustness to noisy inputs, while an extension of compatibility to multilingual text across diverse writing systems. We evaluate the proposed method on the multilingual LAMBADA dataset, WMT24 dataset and the SST-2 benchmark, demonstrating both its resilience to orthographic noise and its effectiveness in multilingual settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。