用历史迭代信息加速AI计算，减少收敛步数。
Accelerating AI Performance using Anderson Extrapolation on GPUs
- 利用历史迭代数据做向量外推，动态调整计算过程。
- 减少收敛所需迭代次数，提升训练与推理效率。
- 适合追求高性能计算的AI系统优化场景。
我们提出一种新方法，通过安德森外推（Anderson extrapolation）——一种基于历史迭代窗口的向量到向量映射技术——加速AI性能。该方法识别出混合惩罚出现的交叉点（图1），聚焦于降低收敛所需的迭代次数，以更少但计算更密集的迭代完成任务，同时在速度与内存使用之间权衡，在准确性和算法稳定性之间取得平衡。实验表明，该方法在训练和推理阶段均带来显著性能提升，推动高性能计算（HPC）领域的可扩展性与效率发展。
原文摘要 · Abstract (English)
We present a novel approach for accelerating AI performance by leveraging Anderson extrapolation, a vector-to-vector mapping technique based on a window of historical iterations. By identifying the crossover point (Fig. 1) where a mixing penalty is incurred, the method focuses on reducing iterations to convergence, with fewer more compute-intensive but generally cacheable iterations, balancing speed and memory usage with accuracy and algorithmic stability, respectively. We demonstrate significant improvements, in both training and inference, motivated by scalability and efficiency extensions to the realm of high-performance computing (HPC).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。