让深度模型像人一样反复推敲，用顶层信息改进底层特征
Contextual Feedback Loops: Amplifying Deep Reasoning with Iterative Top-Down Feedback
- 通过上下文向量将高层预测反馈到早期层，迭代优化特征
- 在多个数据集上提升性能，且反馈过程稳定收敛
- 适合追求模型推理能力提升的研究者与开发者
传统深度网络依赖单向反向传播，忽略了高层预测与低层表示之间的协调。本文提出轻量级的上下文反馈回路（CFLs），将顶层输出以紧凑上下文向量形式重新注入早期层，通过门控适配器实现迭代精炼。该机制在多步反馈中统一前向与反馈推理，使高层输出持续优化低层特征。尽管计算开销极小，CFLs 在 CIFAR-10、ImageNet-1k、SpeechCommands 及 GLUE SST-2 等任务上均获得稳定增益。在满足弱 Lipschitz 条件下，基于 Banach 不动点理论，反馈更新可保证稳定收敛。结果表明，即使微弱的自上而下反馈也能显著增强深度模型，契合认知科学中的迭代感知理论。
原文摘要 · Abstract (English)
Conventional deep networks rely on one-way backpropagation that overlooks reconciling high-level predictions with lower-level representations. We propose \emph{Contextual Feedback Loops} (CFLs), a lightweight mechanism that re-injects top-down context into earlier layers for iterative refinement. Concretely, CFLs map the network's prediction to a compact \emph{context vector}, which is fused back into each layer via gating adapters. Unrolled over multiple feedback steps, CFLs unify feed-forward and feedback-driven inference, letting top-level outputs continually refine lower-level features. Despite minimal overhead, CFLs yield consistent gains on tasks including CIFAR-10, ImageNet-1k, SpeechCommands, and GLUE SST-2. Moreover, by a Banach Fixed Point argument under mild Lipschitz conditions, these updates converge stably. Overall, CFLs show that even modest top-down feedback can substantially improve deep models, aligning with cognitive theories of iterative perception.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。