用一致性蒸馏加速深度均衡模型推理，少步计算仍保高精度。
Consistency Deep Equilibrium Models
- 将迭代求解视为沿固定微分轨迹演化，训练模型直接映射中间状态到平衡点。
- 相同少步推理下，准确率比传统隐式DEQ提升2-20倍。
- 支持多步评估，灵活权衡计算量与性能，适合高效部署场景。
深度均衡模型（DEQ）作为深度学习的新范式，能以恒定内存建模无限深度网络，但其迭代求解导致推理延迟高。本文提出一致性深度均衡模型（C-DEQ），通过一致性蒸馏加速推理。将DEQ的迭代过程建模为沿固定常微分方程（ODE）轨迹向平衡点演进，训练C-DEQ在该轨迹上将中间状态一致地映射至最终固定点，实现少步推理同时保持教师模型性能。此外，支持多步评估，可灵活调整计算量以获取性能增益。跨多个领域任务的大量实验表明，在相同少步推理预算下，C-DEQ相比隐式DEQ实现2-20×的准确率提升。代码已开源：https://github.com/landrarwolf/CDEQ。
原文摘要 · Abstract (English)
Deep Equilibrium Models (DEQs) have emerged as a powerful paradigm in deep learning, offering the ability to model infinite-depth networks with constant memory usage. However, DEQs incur significant inference latency due to the iterative nature of fixed-point solvers. In this work, we introduce the Consistency Deep Equilibrium Model (C-DEQ), a novel framework that leverages consistency distillation to accelerate DEQ inference. We cast the DEQ iterative inference process as evolution along a fixed ODE trajectory toward the equilibrium. Along this trajectory, we train C-DEQs to consistently map intermediate states directly to the fixed point, enabling few-step inference while preserving the performance of the teacher DEQ. At the same time, it facilitates multi-step evaluation to flexibly trade computation for performance gains. Extensive experiments across various domain tasks demonstrate that C-DEQs achieve consistent 2-20$\times$ accuracy improvements over implicit DEQs under the same few-step inference budget. Our code is available at https://github.com/landrarwolf/CDEQ.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。