提出低秩防御与电路引导代理模型，显著降低大模型对抗训练计算开销。
Efficient LLM Adversarial Training via Low-Rank Defense and Circuit-Guided Surrogates
- 在防御侧采用低秩微调，精准定位需保护的词元
- 攻击侧仅提取关键神经路径构建轻量代理模型
- 相比标准方法减少48.1%计算量，参数量仅占0.0118%
对抗训练是抵御对抗攻击最有效的防御手段之一，但其计算成本在现代大模型规模下仍难以承受。尽管已有如潜在对抗训练（LAT）等缓解策略，但仍存在高昂计算开销。本文从防御和攻击双侧出发，系统研究计算效率优化方案：(1) 防御侧优化：探索在LAT中进行表示微调（ReFT），揭示若微调粒度与攻击位置不匹配将引发潜在问题；(2) 攻击侧优化：在每轮LAT迭代中，仅提取大语言模型中的相关电路构建轻量级代理模型，避免在完整模型上执行前向-反向传播以生成对抗样本。本文为两种策略提供理论依据与数值验证。最终，相比全参数微调的标准LAT，本方法平均降低每步对抗训练的浮点运算量（FLOPs）48.1%，且仅需0.0118%的可训练参数。
原文摘要 · Abstract (English)
Adversarial training is one of the most effective defenses against adversarial attacks, yet the computational cost remains prohibitive at modern scales, especially for large language models (LLMs). While existing mitigation strategies, e.g., latent adversarial training (LAT), have been developed, they still incur a high computational cost. In this work, we comprehensively investigate computation-efficient strategies to speed up LAT from two complementary perspectives: (1) Defense-side optimization: We explore the representation fine-tuning (ReFT) within LAT, and reveal a potential issue if there is a mismatch on which tokens to apply ReFT and the attack. (2) Attack-side optimization: When computing adversarial attacks in each LAT iteration, we extract only the relevant circuits from the LLM to construct a lightweight surrogate model, avoiding the computation in the forward-backward passes through the full model during the attack generation. For both perspectives, we provide theoretical justifications and numerical evidence to illustrate the effectiveness of the proposed strategies. Ultimately, compared to standard LAT with full fine-tuning, our method on average reduces per-step adversarial-training FLOPs by 48.1% while requiring only 0.0118% trainable parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。