arXiv:2601.03506cs.CLcs.AI2026-01中稿 · CIKM 2026被引 1

通过动态选择层,让模型根据问题复杂度自动调整推理长度,更快更省算力。

QA-Merging: Query-Adaptive Reasoning via Layer Selective Model Merging

  • 基于激活值识别关键层,只对部分层做微调以适应不同问题
  • 在7个基准上推理成本降低,且准确率保持领先
  • 无需重训练,适合想提升效率的模型部署者

近期的大规模推理模型(LRMs)通过生成长链式思维(Long-CoT)在复杂推理任务中表现优异,但简单问题也生成长推理,造成额外计算与延迟。现有自适应推理方法多依赖重训练或复杂提示,成本高或对提示敏感。模型合并提供更优方案,但现有方法常为静态、输入无关,或需全层校准。为此,本文提出查询自适应层选择性合并(QA-Merging),一种基于激活值的合并框架,无需从头训练或大规模数据,即可融合长推理与短推理模型。该方法构建小规模带标签校准集,为每个查询分配合适推理模式;基于实证分析发现长/短推理行为在各层分布不均,仅对差异大的层进行特征对齐与对比塑形校准,其余层用闭式隐状态修正。在两个模型规模下的7个主流推理基准测试中,该方法显著降低推理开销,同时保持强性能。

原文摘要 · Abstract (English)

Recent large reasoning models (LRMs) have achieved strong performance on complex reasoning tasks by generating a long chain-of-thought (Long-CoT). However, such lengthy reasoning is often unnecessary for simple queries, leading to additional computation and latency. Existing approaches to adaptive reasoning typically rely on retraining the model or designing sophisticated prompting, which are either prohibitively expensive or highly sensitive to the prompt formulation. Model merging provides a more balanced alternative for adaptive reasoning by avoiding expensive training and integrating Long-CoT and Short-CoT behaviors. However, existing merging methods are often static and input-agnostic, or rely on costly all-layer calibration, which limits their effectiveness for query-adaptive reasoning. To tackle these challenges, we propose Query-adaptive Layer Selective Merging (QA-Merging), an activation-based merging framework that integrates a Long-CoT model and a Short-CoT model to obtain a query-adaptive reasoner without training from scratch or requiring large-scale additional data. QA-Merging first constructs a small pattern-labeled calibration set that assigns each query an appropriate reasoning pattern. Motivated by our empirical analysis that Long-CoT and Short-CoT behaviors diverge unevenly across transformer layers, QA-Merging identifies layers with high reasoning pattern divergence and calibrates only these layers through feature alignment and contrastive shaping, while applying closed-form hidden-state correction to the remaining layers. Experiments on seven widely used reasoning benchmarks across two model scales demonstrate that QA-Merging reduces inference cost and maintains strong performance.

推理优化模型合并自适应推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。