通过五种消融实验,揭示扩散模型推理中对齐机制的关键作用。
Retrieval-Warmed Energy-Based Reasoning: A Five-Arm Ablation Methodology for Diffusion-as-Inference on Structured Reasoning Tasks
- 设计五种消融策略,分离出初始值对齐、随机启动和先验偏差的影响。
- 在图连通性任务中,对齐机制使准确率提升35个百分点。
- 适用于需要可解释性推理的结构化任务研究者。
我们研究了检索预热的能量基推理(RW-EBR),这是一种基于IRED能量模型并引入现代霍普菲尔德轨迹记忆的扩散模型。提出一种五臂消融方法(预言、最佳常数、每查询随机、打乱、对齐),以解耦三个混淆因素:类别先验偏移、随机预热和图对齐值重用。该诊断分解借鉴自LLM-RAG评估方法。在连通性-2(Erdős–Rényi全对可达性)任务上,固定值分布与检索机制,仅破坏每图对齐时,对齐与打乱的预言结果差距达+35个百分点平衡准确率,在1000张图的验证集上表现显著;而每查询随机初始化反而低于冷启动。部署中的冷启动预测管道因存储值质量未达标而失败。相同诊断逻辑应用于带专用键编码器的数独任务,揭示关键阻塞环节为键质量,而非对齐。该框架为结构化与时空推理任务提供故障模式可解释性分析工具。
原文摘要 · Abstract (English)
Warm-started diffusion samplers accelerate iterative inference, but it is rarely clear which part of the pipeline carries the gain. We study \textbf{retrieval-warmed energy-based reasoning (RW-EBR)} -- an IRED energy-based diffusion model \cite{du2024ired} augmented with a Modern Hopfield trajectory memory -- and contribute a \textbf{five-arm ablation methodology} (oracle, best-constant, per-query-random, shuffled, aligned) that separates three confounded effects: class-prior bias shift, stochastic warm-starting, and graph-aligned value reuse. The diagnostic decomposition is adapted from LLM-RAG evaluation \cite{ru2024ragchecker}. On \textbf{connectivity-2} (Erdős--Rényi all-pairs reachability), the aligned-vs-shuffled-oracle swing reaches \textbf{$+35$\,pp} balanced accuracy on a fixed 1{,}000-graph validation-set diagnostic, with value distribution and retrieval mechanics fixed, only per-graph alignment destroyed, while per-query random initialisation falls below cold -- per-graph alignment, not bias shift or stochasticity, dominates. Yet the \emph{deployable} cold-prediction pipeline misses the acceptance gate at stored-value quality. The same diagnostic logic, stopped at the key-quality screen, applied to \textbf{Sudoku} with a task-specific key encoder produces a clean negative at a \emph{different} component -- key quality, under the current setup. The decomposition names the first blocking component on each task. The setting -- graph reachability refined by an iterative diffusion sampler, with explainability of failure modes as the lens -- places the work within structured and spatio-temporal reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。