通过架构分离推理与非推理路径,解决大模型思维模式混淆问题。
Path-Lock Expert: Separating Reasoning Mode in Hybrid Thinking via Architecture-Level Separation
- 将每个解码层的MLP拆分为专用于推理和非推理的两个专家模块。
- 在AIME24上使非推理模式反射性文本减少17倍,输出长度减半,准确率提升9.3个百分点。
- 适合需要精准控制推理行为的数学、科学问答场景,如教育评测系统。
混合思维语言模型虽具备显式的 /think 与 /no_think 模式,但现有设计未能实现清晰分离。即使在 /no_think 模式下,模型仍常生成长篇自省内容,导致推理泄漏。现有方法通过数据优化与多阶段训练缓解此问题,但因两种模式共享相同的前馈参数,泄漏仍存在。本文提出路径锁定专家(Path-Lock Expert, PLE),从架构层面解决该问题:将每个解码层的单个MLP替换为两个语义锁定的专家,分别对应 /think 与 /no_think 模式,同时共享注意力、嵌入、归一化及语言模型头。一个确定性控制令牌路由器为整个序列选择唯一专家路径,确保推理保持密集模型的逐标记计算模式,并在监督微调中实现模式纯净的更新。在数学与科学推理基准测试中,PLE在维持强 /think 性能的同时,显著增强模式分离效果。以 Qwen3-4B 为例,在 AIME24 上相比 SFT-only 基线,/no_think 模式生成的反思性令牌减少 17 倍(6.01 vs. 0.35 每答案),输出长度缩短至 4101 对比 8665 词,准确率从 35.33% 提升至 44.67%,而 /think 模式性能保持稳定(61.33% vs. 60.00%)。结果表明,可控混合思维本质上是架构问题,分离模式专属前馈路径是一种简单有效的解决方案。
原文摘要 · Abstract (English)
Hybrid-thinking language models expose explicit /think and /no_think modes, but current designs do not separate them cleanly. Even in /no_think mode, models often emit long and self-reflective responses, causing reasoning leakage. Existing work reduces this issue through better data curation and multi-stage training, yet leakage remains because both modes are still encoded in the same feed-forward parameters. We propose Path-Lock Expert (PLE), an architecture-level solution that replaces the single MLP in each decoder layer with two semantically locked experts, one for /think and one for /no_think, while keeping attention, embeddings, normalization, and the language-model head shared. A deterministic control-token router selects exactly one expert path for the entire sequence, so inference preserves the dense model's per-token computation pattern and each expert receives mode-pure updates during supervised fine-tuning. Across math and science reasoning benchmarks, PLE maintains strong /think performance while producing a substantially stronger mode separation, a /no_think mode with higher accuracy and far less reasoning leakage. On Qwen3-4B, for example, compared to the SFT-only baseline on AIME24, PLE generates 17x fewer reflective tokens (6.01 vs. 0.35 per answer) and 2x shorter outputs (8665 vs. 4101 tokens), and improves /no_think accuracy from 35.33% to 44.67%, while maintaining /think-mode performance (61.33% vs. 60.00%). These results suggest that controllable hybrid thinking is fundamentally an architectural problem, and separating mode-specific feed-forward pathways is a simple and effective solution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。