arXiv:2604.23877cs.CL2026-04ACL

发现大模型中三类逻辑推理可表示为独立向量,提出互补优化方法提升性能。

Knowledge Vector of Logical Reasoning in Large Language Models

论文配图:Knowledge Vector of Logical Reasoning in Large Language Models
图 1 · 摘自论文原文
  • 将三类逻辑推理建模为线性空间中的特定知识向量
  • 引入互补损失与子空间约束,使向量间相互促进且不丢失特性
  • 实验表明优化后推理能力显著提升,适合研究模型推理机制者阅读

逻辑推理是大语言模型的核心能力,主要包括演绎、归纳和溯因三种形式。本文研究这些推理类型在大模型中的知识表征,并分析其相互关系。结果表明,每种推理均可在低维线性空间中以特定知识向量表示,但各向量间基本独立。受认知科学理论启发,结合观察到的跨类型推理链可相互受益的现象,我们提出一种互补子空间约束的精炼框架:通过互补损失让各推理向量利用其他类型辅助知识,同时通过子空间约束损失保留各自独特特征。沿推理向量的控制实验显示,融合互补知识的优化向量带来一致性能提升。此外,对各推理向量的机制可解释性分析揭示了不同推理间的共性与差异特征。

原文摘要 · Abstract (English)

Logical reasoning serve as a central capability in LLMs and includes three main forms: deductive, inductive, and abductive reasoning. In this work, we study the knowledge representations of these reasoning types in LLMs and analyze the correlations among them. Our analysis shows that each form of logical reasoning can be captured as a reasoning-specific knowledge vector in a linear representation space, yet these vectors are largely independent of each other. Motivated by cognitive science theory that these subforms of logical reasoning interact closely in the human brain, as well as our observation that the reasoning process for one type can benefit from the reasoning chain produced by another, we further propose to refine the knowledge representations of each reasoning type in LLMs to encourage complementarity between them. To this end, we design a complementary subspace-constrained refinement framework, which introduces a complementary loss that enables each reasoning vector to leverage auxiliary knowledge from the others, and a subspace constraint loss that prevents erasure of their unique characteristics. Through steering experiments along reasoning vectors, we find that refined vectors incorporating complementary knowledge yield consistent performance gains. We also conduct a mechanism-interpretability analysis of each reasoning vector, revealing insights into the shared and specific features of different reasoning in LLMs.

逻辑推理知识向量模型优化可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。