arXiv:2608.00720cs.ARcs.LG2026-08

针对FPGA推理中带宽瓶颈,提出分步精炼的流式推理框架。

CascadeLUT: Information-Ordered Streaming Inference for Bandwidth-Constrained FPGAs

论文配图:CascadeLUT: Information-Ordered Streaming Inference for Bandwidth-Constrained FPGAs
图 1 · 摘自论文原文
  • 输入特征分批有序传输,逐层逐步提升预测精度
  • 相比基线延迟降低4.0~12.5倍,吞吐提升3.0~5.0倍
  • 适合资源受限设备上实时低功耗神经网络推理

将神经网络映射到FPGA可实现低延迟、低功耗推理,尤其适用于无需乘法器的查找表(LUT)模型。现有方法通常假设数据完整可用,导致在带宽受限的流式场景中出现流水线阻塞。此时瓶颈从计算转向数据传输,大容量输入传输限制了吞吐和能效。本文提出CascadeLUT,一种围绕带宽约束设计的信息结构化推理框架。不缓存全部输入,而是将特征划分为有序子集,随着子集到达逐步优化预测结果。级联结构静态控制各层何时使用输入特征,实现确定性流式推理且无需运行时分支。通过特征调度与硬件数据流协同设计,显著减少数据移动量。在多个数据集上,相比现有LUT基线,延迟降低4.0至12.5倍,吞吐提升3.0至5.0倍,每样本能耗降低最多13.8倍,所用LUT数量为最小DWN基线的1.2至4.4倍。还展示了与LUT推理集成的端侧输入量化,并在真实任务上实现5倍的量化开销降低。

原文摘要 · Abstract (English)

Mapping neural networks to FPGAs enables low-latency, energy-efficient inference, particularly for lookup table (LUT)-based models that eliminate multipliers and map directly to reconfigurable fabric. While prior work achieves high compute efficiency, it typically assumes full-sample availability, causing pipeline stalls in bandwidth-limited streaming scenarios. Here, the bottleneck shifts from computation to data movement, as large input transfers limit throughput and energy efficiency. We present CascadeLUT, an information-structured inference framework organized around bandwidth constraints. Instead of buffering the full input, features are partitioned into ordered subsets and predictions are progressively refined as subsets arrive. The cascade statically controls which layers consume incoming features, enabling deterministic streaming inference without runtime branching. By co-designing feature scheduling with hardware dataflow, CascadeLUT reduces data movement while maintaining accuracy. Across datasets, it achieves 4.0 to 12.5 times lower latency, 3.0 to 5.0 times higher throughput and up to 13.8 times lower energy/sample than prior LUT baselines, using 1.2 to 4.4 times the LUTs of the smallest DWN baseline per task. We also demonstrate on-device input quantization integrated with LUT-based inference and present end-to-end FPGA results on real-world workloads, with 5 times reductions in quantization overhead.

FPGA推理流式计算低功耗LUT优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。