用注意力层生成更自然的对抗样本，测试大模型评估系统可靠性。
Adversarial Lens: Exploiting Attention Layers to Generate Adversarial Examples for Evaluation
- 从注意力层提取词元分布生成对抗扰动，符合模型内部推理过程。
- 在ArgQuality数据集上使评估性能下降12.3%，同时保持语义相似性。
- 适合用于压力测试大模型评估系统，但部分位置扰动会破坏语法。
近期机制可解释性研究发现，中间注意力层编码了逐词元的假设，并逐步优化为最终输出。本文利用这一特性，直接从注意力层的词元分布生成对抗样本。与基于提示或梯度的攻击不同，该方法利用模型内部的词元预测，生成既合理又与模型自身生成过程一致的扰动。我们在使用ArgQuality数据集进行论点质量评估时，以LLaMA-3.1-Instruct-8B作为生成器和评估器，实验表明：基于注意力的对抗样本导致评估性能下降12.3%，且与原始输入语义高度相似。然而，某些层和词元位置的替换会引入语法错误，限制其实际应用效果。总体而言，研究揭示了将中间层表示作为对抗样本来源的潜力与当前局限，为压力测试基于大模型的评估流水线提供了新思路。
原文摘要 · Abstract (English)
Recent advances in mechanistic interpretability suggest that intermediate attention layers encode token-level hypotheses that are iteratively refined toward the final output. In this work, we exploit this property to generate adversarial examples directly from attention-layer token distributions. Unlike prompt-based or gradient-based attacks, our approach leverages model-internal token predictions, producing perturbations that are both plausible and internally consistent with the model's own generation process. We evaluate whether tokens extracted from intermediate layers can serve as effective adversarial perturbations for downstream evaluation tasks. We conduct experiments on argument quality assessment using the ArgQuality dataset, with LLaMA-3.1-Instruct-8B serving as both the generator and evaluator. Our results show that attention-based adversarial examples lead to measurable drops in evaluation performance while remaining semantically similar to the original inputs. However, we also observe that substitutions drawn from certain layers and token positions can introduce grammatical degradation, limiting their practical effectiveness. Overall, our findings highlight both the promise and current limitations of using intermediate-layer representations as a principled source of adversarial examples for stress-testing LLM-based evaluation pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。