用坐标下降法直接优化网络稀疏性，减少ReLU数量提升隐私推理速度。
Coordinate Descent for Network Linearization
- 基于坐标下降法在离散域直接优化，避免平滑近似带来的误差。
- 在多个基准测试上达到当前最优，显著降低ReLU数量同时保持精度。
- 适合需要高效隐私推理的场景，尤其关注计算延迟优化的研究者。
ReLU激活函数是基于ResNet网络的隐私推理中的主要性能瓶颈，因其带来显著的推理延迟。减少ReLU数量是一个离散优化问题，现有主流方法通常采用联合优化网络准确率与ReLU预算的平滑近似策略，但最后的硬阈值步骤常导致较大性能损失。本文提出一种替代方法，直接在离散域中利用坐标下降作为优化框架，从设计上生成稀疏解。通过大量实验验证，该方法在常见基准测试上表现优异，达到当前最优水平。
原文摘要 · Abstract (English)
ReLU activations are the main bottleneck in Private Inference that is based on ResNet networks. This is because they incur significant inference latency. Reducing ReLU count is a discrete optimization problem, and there are two common ways to approach it. Most current state-of-the-art methods are based on a smooth approximation that jointly optimizes network accuracy and ReLU budget at once. However, the last hard thresholding step of the optimization usually introduces a large performance loss. We take an alternative approach that works directly in the discrete domain by leveraging Coordinate Descent as our optimization framework. In contrast to previous methods, this yields a sparse solution by design. We demonstrate, through extensive experiments, that our method is State of the Art on common benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。