通过优化寻找更稳定的卷积变换，让低精度计算下的高效卷积成为可能。
NOVA: Discovering Well-Conditioned Winograd Transforms through Numerical Optimization of Vandermonde Arithmetic
- 将点选择视为连续优化问题，用进化策略搜索更优的分数点配置。
- F(8,3)变换条件数改善415倍，2D下提升超17万倍，恢复高精度推理。
- 无需重训练即可在FP16下恢复模型准确率，适合部署于新型硬件。
Winograd卷积是高效推理的标准算法，可将3×3核的算术复杂度降低2.25倍。然而在低精度计算时代面临关键挑战：数值不稳定性。当分块规模增大以提升效率（如F(6,3)、F(8,3)）时，传统整数变换的条件数急剧上升，达到F(8,3)时κ = 2 × 10^5，导致在FP16或Int8下无法使用。本文提出NOVA（Vandermonde算术的数值优化），打破几十年来基于整数插值的惯例。将Winograd点选择建模为连续优化问题，通过进化策略在R^n−1流形上搜索，候选解经舍入为简单有理数，并通过符号验证确保正确性。该方法发现了一系列稳定且非整数的配置，如{±5/6, ±7/6, ±3/5}，突破传统约束。结果极具颠覆性：在1D中，F(8,3)的条件数改善415倍，2D下对应提升达172,484倍。在真实场景的FP16 ImageNet推理中，标准变换导致精度骤降至4.7%（如VGG16），而NOVA方案恢复至75–78%，提升超70个百分点，无需重训练、校准或学习参数。这些发现的变换可直接替换，有效释放大尺寸分块卷积的效率潜力，适配下一代硬件。
原文摘要 · Abstract (English)
Winograd convolution is the standard algorithm for efficient inference, reducing arithmetic complexity by 2.25x for 3x3 kernels. However, it faces a critical barrier in the modern era of low precision computing: numerical instability. As tiles scale to maximize efficiency (e.g., F(6,3), F(8,3)), the condition numbers of standard integer based transforms explode, reaching kappa = 2 x 10^5 for F(8,3), rendering them unusable in FP16 or Int8. We introduce NOVA (Numerical Optimization of Vandermonde Arithmetic), a discovery framework that breaks the decades old convention of integer interpolation. Treating Winograd point selection as a continuous optimization problem, NOVA searches the manifold R^n-1 via Evolution Strategy, snaps candidates to simple rationals, and guarantees correctness via symbolic verification. This process uncovers a hidden landscape of stable, fractional configurations such as {+-5/6, +-7/6, +-3/5} that defy traditional vocabulary constraints. The impact is transformative: NOVA improves the conditioning of F(8,3) by 415x in 1D, which squares to a 172,484x improvement for 2D convolution. In real world FP16 ImageNet inference, where standard transforms collapse to random chance (e.g., 4.7 percent accuracy on VGG16), NOVA's points restore full accuracy (75 to 78 percent), recovering over 70 percentage points without retraining, calibration, or learned parameters. These discovered transforms act as drop in replacements, effectively unlocking the efficiency of large tile Winograd convolution for next generation hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。