arXiv:2608.18402math.STcs.DS2026-08

针对噪声不均的线性回归,提出高效算法并揭示计算极限。

Algorithms for adaptive and heteroskedastic linear regression at the computational threshold

  • 设计多项式时间算法处理未知方差的异方差回归
  • 当噪声样本占比小(m≫n¹ᐟ⁴)时误差趋近零,优于传统方法
  • 首次揭示计算效率与统计性能间的潜在瓶颈

研究在标签噪声质量各异且未知条件下的有限样本线性回归问题,聚焦异方差与自适应线性回归模型。观测到 n 组 (X_i, Y_i),其中 Y_i = X_i^⊤β + ε_i,ε_i ∼ N(0, σ_i²),σ_i² 未知。问题难度由满足 σ_i² ≤ 1 的样本数 m 决定(m 越大越易)。提出一种多项式时间估计器,当 m ≫ d³ᐟ⁴n¹ᐟ⁴ 时,误差率可达 Õ((nd³/m⁴)¹ᐟ⁶),并给出近乎匹配的下界。当 d = O(1) 时,只要 m ≫ n¹ᐟ⁴,误差即趋于 0,而 L₁ 回归等传统方法需 m ≫ n¹ᐟ²。在自适应回归中,ε_i 独立同分布于未知分布 p,目标是设计通用估计器逼近已知 p 的最优定制估计器。我们构造一非多项式时间估计器,在 p 为至多 k 个对称对数凹密度混合时,表现接近已知 p 且拥有 Õ(n/k) 样本的最优估计器。当 k=1 时,通过数据相关选择 q 可实现多项式时间估计。为探究两问题的计算边界,引入“植入线性回归”模型:X_i ∼ N(0, I_d),m 个样本无噪声,其余 ε_i ∼ N(0,1)。推测在恢复 β 误差 < √(d/n) 时存在信息-计算差距,从 m=d+1 到 m∼d³ᐟ⁴n¹ᐟ⁴ 之间,这由近似匹配的多项式时间算法与统计查询(SQ)下界所暗示。

原文摘要 · Abstract (English)

We study finite-sample linear regression in the presence of varied and unknown label noise, focusing on the heteroskedastic and adaptive linear regression models. Heteroskedastic linear regression models settings where the labels are of varying quality. We receive $n$ pairs $(X_i,Y_i)$ with labels $Y_i=X_i^\topβ+\varepsilon_i$, where $\varepsilon_i\sim N(0,σ_i^2)$ and the variances are unknown to the estimator. One natural measurement of the difficulty of this problem is the number of samples $m$ for which $σ_i^2\le1$ (larger $m$ is easier). We obtain a polynomial-time estimator with rate $\tilde{O}((nd^3/m^4)^{1/6})$ when $m\gg d^{3/4}n^{1/4}$, as well as nearly-matching lower bounds. For $d=O(1)$, our estimator achieves error $o(1)$ when $m\gg n^{1/4}$, whereas $L_1$ regression and other traditional approaches require $m\gg n^{1/2}$. In adaptive linear regression, the errors are drawn i.i.d. from an unknown distribution $p$, and our goal is to design a generic estimator that performs nearly as well as the best custom estimator that knows $p$. We introduce a (computationally inefficient) adaptive estimator that, so long as $p$ is a mixture of $k$ symmetric log-concave densities, achieves error comparable with the optimal estimator that knows $p$ and has $\tildeΘ(n/k)$ samples. For $k=1$, we show that $L_q$ regression (with data-dependent $q$) gives a polynomial-time estimator. Finally, to study the computational limits of both problems, we introduce the planted linear regression problem, where $X_i\sim N(0,I_d)$, $m$ unknown samples are noiseless, and the rest have error $\varepsilon_i\sim N(0,1)$. We conjecture that recovering $β$ up to error $\ll\sqrt{d/n}$ (or exactly) may have an information-computation gap between $m=d+1$ and $m\sim d^{3/4}n^{1/4}$, as is suggested by our near-matching polynomial-time estimator and statistical query (SQ) lower bound.

线性回归异方差计算阈值统计学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。