arXiv:2605.07307cs.CL2026-05被引 1

模型推理链可乱序且稀疏,仍能准确提取答案

Rethinking Dense Sequential Chains: Reasoning Language Models Can Extract Answers from Sparse, Order-Shuffling Chain-of-Thoughts

  • 打乱推理步骤顺序,精度下降极小,证明顺序不重要
  • 只保留关键词和数字,精度反而提升4.7个百分点
  • 即使去掉所有自然语言、随机打乱,仍保持83%准确率

当前推理型语言模型生成密集、顺序依赖的思维链,隐含假设每个词元都重要且必须按序处理。本文通过移除、掩码、打乱和噪声注入等系统性干预,在三个模型和三个基准上对生成的推理链进行测试。结果出人意料:顺序无关性——行级打乱精度下降不足0.5个百分点,词级打乱保留62%-89%精度,仅词元级打乱降至接近零;信息冗余性——掩码数值使精度降为0%,而掩码字母文本反而提升4.7个百分点;鲁棒性——完全去除自然语言并任意打乱,仍达83%精度,注入三倍假答案也未改变精度(83.3%→83.3%),否定频率依赖提取假设。研究揭示答案提取基于稀疏、无序、结构鲁棒的信息基底,为并行化与高效推理生成开辟新路径。

原文摘要 · Abstract (English)

Modern reasoning language models generate dense, sequential chain-of-thought traces implicitly assuming that every token contributes and that steps must be consumed in order. We challenge both assumptions through a systematic intervention pipeline--removal, masking, shuffling, and noise injection--applied to model-generated reasoning chains across three models and three benchmarks. Our findings are counterintuitive on three dimensions. Order: Does the sequential order of a reasoning chain matter for answer extraction? No--line-level shuffling reduces accuracy by less than 0.5 pp; word-level shuffling retains 62%-89% accuracy; only token-level shuffling collapses to near zero. Pretrained-only and instruction-tuned variants exhibit near-identical tolerance (78.67% vs. 78.00% under line shuffling), indicating order-independence originates from pretraining rather than reasoning-specific fine-tuning. Dense: Is all the information in a reasoning chain important for answer extraction? No--masking numeric digits collapses accuracy to exactly 0%, while masking alphabetic prose improves accuracy by 4.7 pp. Robustness: Is a reasoning chain that is both order-shuffling and non-dense still robust? Yes--the most aggressively reduced representation (all natural language removed, lines arbitrarily shuffled) still achieves 83% accuracy, and injecting false answers at 3x true-answer frequency leaves accuracy unchanged (83.3%->83.3%), falsifying a frequency-based extraction account. These results establish that answer extraction operates on a sparse, order-insensitive, and structurally robust informational substrate, opening paths toward parallelized and token-efficient reasoning generation.

推理链稀疏性顺序无关高效生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。