让大模型在多问题中自动压缩推理，提升效率
ConPress: Learning Efficient Reasoning from Multi-Question Contextual Pressure
- 用多问题同时提问诱导模型自我压缩推理过程
- 仅用8000样本，推理耗时减少59%至33%
- 无需外部指导或人工干预，适合部署优化
大型推理模型通常通过生成长链式思维（CoT）来解决复杂任务，带来显著的推理开销。我们发现一种可复现的推理期现象——自压缩：当多个独立且可回答的问题被置于同一提示中时，模型会自发为每个问题生成更短的推理轨迹。该现象源于生成过程中的多问题上下文压力，在不同模型和基准上均稳定出现。基于此，我们提出ConPress（从上下文压力中学习），一种轻量级自监督微调方法。ConPress构建多问题提示以诱导自压缩，采样并解析过滤各问题的推理轨迹，获得简洁且正确的推理路径。这些路径直接用于监督微调，使模型在单问题场景中内化压缩推理行为，无需外部教师、人工修剪或强化学习。仅用8000个微调样本，ConPress在MATH500上将推理令牌使用减少59%，在AIME25上减少33%，同时保持竞争力的准确率。
原文摘要 · Abstract (English)
Large reasoning models (LRMs) typically solve reasoning-intensive tasks by generating long chain-of-thought (CoT) traces, leading to substantial inference overhead. We identify a reproducible inference-time phenomenon, termed Self-Compression: when multiple independent and answerable questions are presented within a single prompt, the model spontaneously produces shorter reasoning traces for each question. This phenomenon arises from multi-question contextual pressure during generation and consistently manifests across models and benchmarks. Building on this observation, we propose ConPress (Learning from Contextual Pressure), a lightweight self-supervised fine-tuning approach. ConPress constructs multi-question prompts to induce self-compression, samples the resulting model outputs, and parses and filters per-question traces to obtain concise yet correct reasoning trajectories. These trajectories are directly used for supervised fine-tuning, internalizing compressed reasoning behavior in single-question settings without external teachers, manual pruning, or reinforcement learning. With only 8k fine-tuning examples, ConPress reduces reasoning token usage by 59% on MATH500 and 33% on AIME25, while maintaining competitive accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。