垂直排列文本会严重降低大模型分类准确率,且思维链无效。
Vulnerability of LLMs to Vertically Aligned Text Manipulations
- 测试垂直排列文本对大模型的影响,发现普遍性能下降。
- 垂直输入使模型准确率显著降低,最高降幅达35%。
- 适合关注模型安全与对抗样本的研究者阅读。
垂直文本输入在数学计算和文字数独等实际应用中常见。尽管当前大语言模型(LLMs)在自然语言任务中表现优异,但仍易受文本格式变化影响。近期研究显示,对编码器类模型进行垂直对齐文本输入,可显著降低其在文本分类任务中的准确率。虽然人类能轻松理解此类输入,但模型却容易被误导,可能在涉及有害或敏感信息的场景中绕过检测。随着大模型广泛应用,一个关键问题浮现:解码器类大模型是否也存在类似脆弱性?本文系统研究了多种大模型在多个文本分类数据集上对垂直文本输入的响应,并分析其成因。结果表明:(i) 垂直文本输入显著降低大模型在文本分类任务中的准确率;(ii) 思维链(Chain-of-Thought, CoT)推理无法帮助模型识别垂直输入或缓解其脆弱性,但经过精心设计的少样本学习可有效改善;(iii) 通过分析分词机制与注意力矩阵,揭示了该脆弱性的内在原因。
原文摘要 · Abstract (English)
Vertical text input is commonly encountered in various real-world applications, such as mathematical computations and word-based Sudoku puzzles. While current large language models (LLMs) have excelled in natural language tasks, they remain vulnerable to variations in text formatting. Recent research demonstrates that modifying input formats, such as vertically aligning words for encoder-based models, can substantially lower accuracy in text classification tasks. While easily understood by humans, these inputs can significantly mislead models, posing a potential risk of bypassing detection in real-world scenarios involving harmful or sensitive information. With the expanding application of LLMs, a crucial question arises: Do decoder-based LLMs exhibit similar vulnerabilities to vertically formatted text input? In this paper, we investigate the impact of vertical text input on the performance of various LLMs across multiple text classification datasets and analyze the underlying causes. Our findings are as follows: (i) Vertical text input significantly degrades the accuracy of LLMs in text classification tasks. (ii) Chain-of-Thought (CoT) reasoning does not help LLMs recognize vertical input or mitigate its vulnerability, but few-shot learning with careful analysis does. (iii) We explore the underlying cause of the vulnerability by analyzing the inherent issues in tokenization and attention matrices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。