通过重塑神经电路,精准修复大模型的推理错误而不影响其他能力。
Reforming the Mechanism: Editing Reasoning Patterns in LLMs with Circuit Reshaping
- 提出电路重构方法,分离重叠的推理神经回路以减少干扰。
- 在三个难度级别的逻辑推理任务中,生成性与局部性均优于基线。
- 适合需要精准优化推理能力且不破坏原有功能的研究者使用。
大型语言模型常表现出缺陷推理能力,影响可靠性。现有方法将推理视为整体技能,训练泛化但效率低,难以定位特定错误。本文提出推理编辑范式,可选择性修改特定推理模式,同时保留其他推理路径。该任务存在泛化性与局部性之间的根本权衡:编辑是否能推广到同类推理任务,同时不损害其他能力。系统研究发现电路干扰定律:不同推理模式间的编辑干扰与其神经电路重叠度成正比。基于此,提出REdit框架,首次在编辑前主动重塑神经电路,调节干扰并缓解权衡。REdit包含三部分:(i) 对比式电路重构,通过解耦重叠电路直接应对泛化-局部性矛盾;(ii) 元对比学习,将可迁移性扩展至新推理模式;(iii) 双层保护机制,通过约束重构方向和正则化任务预测,保护已有能力。在Qwen-2.5-3B上对三类难度的命题逻辑推理任务进行大量实验,结果表明REdit始终在泛化性和局部性上优于基线,数学推理验证也显示更广泛应用潜力。代码已开源于https://github.com/LzyFischer/REdit。
原文摘要 · Abstract (English)
Large language models (LLMs) often exhibit flawed reasoning ability that undermines reliability. Existing approaches to improving reasoning typically treat it as a general and monolithic skill, applying broad training which is inefficient and unable to target specific reasoning errors. We introduce Reasoning Editing, a paradigm for selectively modifying specific reasoning patterns in LLMs while preserving other reasoning pathways. This task presents a fundamental trade-off between Generality, the ability of an edit to generalize across different tasks sharing the same reasoning pattern, and Locality, the ability to preserve other reasoning capabilities. Through systematic investigation, we uncover the Circuit-Interference Law: Edit interference between reasoning patterns is proportional to the overlap of their neural circuits. Guided by this principle, we propose REdit, the first framework to actively reshape neural circuits before editing, thereby modulating interference between reasoning patterns and mitigating the trade-off. REdit integrates three components: (i) Contrastive Circuit Reshaping, which directly addresses the generality-locality trade-off by disentangling overlapping circuits; (ii) Meta-Contrastive Learning, which extends transferability to novel reasoning patterns; and (iii) Dual-Level Protection, which preserves preexisting abilities by constraining reshaping update directions and regularizing task-level predictions. Extensive experiments with Qwen-2.5-3B on propositional logic reasoning tasks across three difficulty levels demonstrate that REdit consistently achieves superior generality and locality compared to baselines, with additional validation in mathematics showing broader potential. Our code is available at https://github.com/LzyFischer/REdit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。