提出统一框架,分析随机压缩下的双线性形式,提升算法精度与效率。
Beyond Johnson-Lindenstrauss: Uniform Bounds for Sketched Bilinear Forms
- 基于泛化链技术,建立双线性形式的统一分析框架。
- 在多矩阵压缩下,误差随√T增长,优于传统方法。
- 适用于联邦学习与强化学习,依赖几何复杂度而非高维维度。
随机压缩下的向量或矩阵内积的统一界是机器学习与随机算法中多项重要结果的基础,包括Johnson-Lindenstrauss引理、受限等距性质(RIP)、随机压缩与近似线性代数。然而,许多现代分析涉及*压缩双线性形式*,现有统一界要么不适用,要么在一般集合上不够紧致。本文提出一个通用框架来分析此类压缩双线性形式,并以相关集合的几何复杂度为指标导出统一界。方法基于泛化链,引入处理两组集合上确界的新型技术。进一步将结果扩展至包含T个独立压缩矩阵的情形,证明偏差按√T增长。该统一分析可恢复J-L引理等经典结果,并推广RIP型保证。此外,获得压缩联邦学习算法的改进收敛界,其中交叉项自然出现于压缩梯度;设计了压缩版带宽算法,其遗憾界依赖动作集与参数集的几何复杂度,而非环境维度。
原文摘要 · Abstract (English)
Uniform bounds on sketched inner products of vectors or matrices underpin several important computational and statistical results in machine learning and randomized algorithms, including the Johnson-Lindenstrauss (J-L) lemma, the Restricted Isometry Property (RIP), randomized sketching, and approximate linear algebra. However, many modern analyses involve *sketched bilinear forms*, for which existing uniform bounds either do not apply or are not sharp on general sets. In this work, we develop a general framework to analyze such sketched bilinear forms and derive uniform bounds in terms of geometric complexities of the associated sets. Our approach relies on generic chaining and introduces new techniques for handling suprema over pairs of sets. We further extend these results to the setting where the bilinear form involves a sum of $T$ independent sketching matrices and show that the deviation scales as $\sqrt{T}$. This unified analysis recovers known results such as the J-L lemma as special cases, while extending RIP-type guarantees. Additionally, we obtain improved convergence bounds for sketched Federated Learning algorithms where such cross terms arise naturally due to sketched gradient compression, and design sketched variants of bandit algorithms with sharper regret bounds that depend on the geometric complexity of the action and parameter sets, rather than the ambient dimension.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。