用可执行错误操作符修复语言模型推理中的语义错误。
Semantic Reasoning Denoising: Correcting Language Model Reasoning with Semantic Operators

- 设计可执行错误操作符,精准描述错误类型与位置。
- 在6个基准上平均提升3.2分,跨数据集仍优于强基线。
- 适合需要高可靠推理的数学、代码与常识任务场景。
大语言模型生成的推理过程虽流畅,但局部语义错误会传播至错误结论,而无约束的自修正可能保留、放大或引入新错误。现有扩散语言模型通常将噪声定义为词元掩码或替换,而非推理过程中的错误。本文提出语义推理去噪(SRD),一种基于操作符的马尔可夫去噪方法,用于自然语言推理轨迹。SRD用可执行错误操作符表示语义噪声,描述错误类型、位置及受损与修复的命题。组合这些操作符可构建渐进式更嘈杂状态。训练时,模型学习识别当前轨迹中的活跃语义噪声,并重建相邻低噪声状态。推理时,噪声感知去噪反复预测逆向操作符并验证其适用性,每次执行都使轨迹向稳定状态局部推进。在涵盖数学、代码、知识和常识的六个域内基准上,SRD平均优于最强同架构基线3.2分;在七个跨数据集迁移任务中,表现媲美Llama-3-8B-Instruct,且平均优于最强Qwen3-8B基线2.9分。对噪声源、目标函数与去噪深度的分析进一步表明,结构化语义噪声预测与迭代操作执行是性能提升的关键。
原文摘要 · Abstract (English)
Large language models can produce fluent reasoning traces whose local semantic errors propagate to an incorrect conclusion, while unconstrained self-correction may preserve, amplify, or introduce errors. Existing diffusion language models provide iterative refinement, but usually define noise as token masking or replacement rather than as errors in the reasoning process. We present Semantic Reasoning Denoising (SRD), an operatorized Markov denoising method for natural-language reasoning trajectories. SRD represents semantic noise with executable error operators that describe the error type, its location, and the corrupted and repaired propositions. Composing these operators constructs progressively noisier states. During training, the model learns to identify the semantic noise active in the current trajectory and to reconstruct the paired adjacent lower-noise state. During inference, noise-level-aware denoising repeatedly predicts an inverse operator and checks whether it is applicable, so each executed update makes a localized move toward a stable trajectory. Across six in-domain benchmarks spanning mathematics, code, knowledge, and commonsense, SRD improves the strongest same backbone baseline by 3.2 points on average. On seven cross-dataset transfer targets, it remains competitive with Llama-3-8B-Instruct and improves the strongest Qwen3-8B baseline average by 2.9 points. Analyses of noise sources, objectives, and denoising depth further show that structured semantic-noise prediction and iterative operator execution are central to the improvement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。