提出段级鲁棒性框架，让模型在提示微调时更抗干扰。
Where Do Prompt Perturbations Break Generation? A Segment-Level View of Robustness in LoRA-Tuned Language Models
- 将生成文本拆解为语义段，定位关键信息漂移
- 在多种扰动下显著提升生成稳定性，跨数据集迁移更强
- 适合关注模型输出可靠性与泛化能力的研究者
大语言模型对提示的微小变化敏感，现有鲁棒性方法多在整句层面保证一致性，却可能掩盖关键实体、关系或结论的偏差。本文提出 S²R²，一种基于段级的鲁棒性微调框架：将干净与扰动生成结果分解为语义段，通过最优传输对齐，并惩罚意义漂移最大的段。为连接输出目标与模型适应，引入基于注意力重分配的适配器稳定性正则项，以 LoRA 范数控制作为扰动放大证据转移的代理。从 PAC-Bayesian 复杂度视角看，控制适配器增长有助于超越观测扰动的泛化。在摘要任务上实验表明，S²R² 在拼写错误、删除、同义替换和改写等扰动下均提升鲁棒性，同时保持良好原始性能，且跨数据集迁移能力优于基于一致性的基线。
原文摘要 · Abstract (English)
Large language models are sensitive to minor prompt perturbations, yet existing robustness methods usually enforce consistency at the whole-sequence level. This holistic view can hide an important failure mode: a perturbed response may remain globally similar to the clean one while drifting on a critical entity, relation, or conclusion. We introduce S$^2$R$^2$, a segment-level framework for robust LoRA fine-tuning. S$^2$R$^2$ decomposes clean and perturbed generations into semantic segments, aligns them with an optimal-transport objective, and penalises the segments with the largest meaning drift. To connect this output-side objective with model adaptation, we add an adapter-stability regulariser motivated by segment-level attention reallocation, using LoRA norm control as a tractable proxy for limiting perturbation-amplified evidence shifts. A PAC-Bayesian complexity view further explains why controlling adapter growth may support transfer beyond observed perturbations. Experiments on summarisation benchmarks show that S$^2$R$^2$ improves robustness under typographical noise, deletion, synonym replacement, and paraphrasing, while maintaining competitive clean performance and stronger cross-dataset transfer than consistency-based baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。