揭示思维链对输入扰动的敏感性边界,发现推理步数越多越不稳定。
Bounds of Chain-of-Thought Robustness: Reasoning Steps, Embed Norms, and Beyond
- 通过理论推导给出输入扰动上限,与推理步骤数正相关
- 证明即使无限推理也无法消除扰动影响,且嵌入向量范数越高越稳定
- 在主流模型和数据集上验证理论,适用于提升提示优化可靠性
现有研究显示,思维链(CoT)输出受输入扰动显著影响。尽管许多方法试图通过优化提示减轻该影响,但缺乏对扰动如何影响CoT输出的理论解释,限制了对推理过程中扰动传播的理解,也阻碍了提示优化方法的进一步改进。本文首次从理论上分析输入扰动对CoT输出波动的影响。我们推导出在输出波动可接受范围内的输入扰动上界,并证明:(i) 该上界与思维链中的推理步骤数正相关;(ii) 即使推理过程无限延长,也无法消除输入扰动的影响。随后,我们将结论应用于线性自注意力(LSA)模型——一种简化的Transformer结构,证明其输入扰动上界与输入嵌入及隐藏状态向量的范数负相关。为验证理论分析,我们在三个主流数据集和四个主流模型上开展实验,结果与理论预测高度一致,实证支持了本研究结论。
原文摘要 · Abstract (English)
Existing research indicates that the output of Chain-of-Thought (CoT) is significantly affected by input perturbations. Although many methods aim to mitigate such impact by optimizing prompts, a theoretical explanation of how these perturbations influence CoT outputs remains an open area of research. This gap limits our in-depth understanding of how input perturbations propagate during the reasoning process and hinders further improvements in prompt optimization methods. Therefore, in this paper, we theoretically analyze the effect of input perturbations on the fluctuation of CoT outputs. We first derive an upper bound for input perturbations under the condition that the output fluctuation is within an acceptable range, based on which we prove that: (i) This upper bound is positively correlated with the number of reasoning steps in the CoT; (ii) Even an infinitely long reasoning process cannot eliminate the impact of input perturbations. We then apply these conclusions to the Linear Self-Attention (LSA) model, which can be viewed as a simplified version of the Transformer. For the LSA model, we prove that the upper bound for input perturbation is negatively correlated with the norms of the input embedding and hidden state vectors. To validate this theoretical analysis, we conduct experiments on three mainstream datasets and four mainstream models. The experimental results align with our theoretical analysis, empirically demonstrating the correctness of our findings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。