arXiv:2604.19488cs.AI2026-04

让大模型跨领域推理更准,通过隐藏层干预对齐思维模式。

CoDA: Towards Effective Cross-domain Knowledge Transfer via CoT-guided Domain Adaptation

  • 在隐藏层用轻量适配器对齐源域与目标域的推理表示。
  • 在多个逻辑推理任务上显著超越现有最佳方法。
  • 特别适合低资源科学、法律等缺乏高质量示例的领域。

大语言模型在逻辑推理方面虽有显著进展,但仍远未达到人类水平。上下文学习通过提供专家精心设计的领域内样例来提升模型表现,但在低资源科学、新兴生物医学领域或小众法律管辖区域等真实场景中,高质量的领域内示例稀缺甚至缺失,限制了该方法的适用性。为缓解此问题,近期研究尝试检索跨领域样本作为替代示例,但提升效果有限。这主要归因于源域与目标域间显著的分布差异,阻碍模型有效识别并利用共享结构或潜在推理模式。因此,仅依赖原始文本提示时,大模型难以稳健系统地抽象和迁移跨领域知识。为此,我们提出 CoDA,通过轻量级适配器直接干预中间隐藏状态。结合基于思维链(CoT)增强的参考表示特征蒸馏与核化最大均值差异(MMD)进行分布匹配,实现源域与目标域隐式推理表示的对齐。在多种模型族的多个逻辑推理任务上的大量实验验证了 CoDA 的有效性,显著优于先前最先进基线。

原文摘要 · Abstract (English)

Large language models (LLMs) have achieved substantial advances in logical reasoning, yet they continue to lag behind human-level performance. In-context learning provides a viable solution that boosts the model's performance via prompting its input with expert-curated, in-domain exemplars. However, in many real-world, expertise-scarce domains, such as low-resource scientific disciplines, emerging biomedical subfields, or niche legal jurisdictions, such high-quality in-domain demonstrations are inherently limited or entirely unavailable, thereby constraining the general applicability of these approaches. To mitigate this limitation, recent efforts have explored the retrieval of cross-domain samples as surrogate in-context demonstrations. Nevertheless, the resulting gains remain modest. This is largely attributable to the pronounced domain shift between source and target distributions, which impedes the model's ability to effectively identify and exploit underlying shared structures or latent reasoning patterns. Consequently, when relying solely on raw textual prompting, LLMs struggle to abstract and transfer such cross-domain knowledge in a robust and systematic manner. To address these issues, we propose CoDA, which employs a lightweight adapter to directly intervene in the intermediate hidden states. By combining feature-based distillation of CoT-enriched reference representations with Maximum Mean Discrepancy (MMD) for kernelized distribution matching, our method aligns the latent reasoning representation of the source and target domains. Extensive experimental results on multiple logical reasoning tasks across various model families validate the efficacy of CoDA by significantly outperforming the previous state-of-the-art baselines by a large margin.

大模型跨域迁移逻辑推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。