用网络流方法解决任意采样模式下的矩阵补全,精准量化每处缺失值的估计难度。
Entry-Specific Matrix Estimation under Arbitrary Sampling Patterns through the Lens of Network Flows
- 基于观测数据构建二分图,用网络流(如电流流)设计矩阵补全算法。
- 单个条目估计误差与图中对应边的有效电阻成正比,理论误差有紧界。
- 适用于因果推断,可准确估计个体效应和单位/时间混杂因素,适合高维面板数据。
矩阵补全旨在根据稀疏观测值预测低秩矩阵中的缺失项。通常假设观测模式为均匀随机或特定结构,但对任意采样模式的理解仍不足。本文针对任意采样模式,提出一种基于二分图中网络流的矩阵补全算法。对于加性矩阵,使用电流流并建立每个条目的误差上界,该上界依赖于观测集,且与匹配的极小极大下界一致。结果表明,特定条目恢复的极小极大平方误差与图中对应边的有效电阻成正比。此外,所提估计器等价于最小二乘估计器。将其应用于两向固定效应模型,可准确推断个体因果效应及个体与时间特异性混杂因子。对于秩-1矩阵,利用边不相交路径构造估计器,在采样足够密集时达到极小极大最优。本研究引入一类由网络流参数化的估计器,为各条目估计难度提供了精细且直观的解释,突破了仅依赖全局性能指标的局限。
原文摘要 · Abstract (English)
Matrix completion tackles the task of predicting missing values in a low-rank matrix based on a sparse set of observed entries. It is often assumed that the observation pattern is generated uniformly at random or has a very specific structure tuned to a given algorithm. There is still a gap in our understanding when it comes to arbitrary sampling patterns. Given an arbitrary sampling pattern, we introduce a matrix completion algorithm based on network flows in the bipartite graph induced by the observation pattern. For additive matrices, the particular flow we used is the electrical flow and we establish error upper bounds customized to each entry as a function of the observation set, along with matching minimax lower bounds. Our results show that the minimax squared error for recovery of a particular entry in the matrix is proportional to the effective resistance of the corresponding edge in the graph. Furthermore, we show that our estimator is equivalent to the least squares estimator. We apply our estimator to the two-way fixed effects model and show that it enables us to accurately infer individual causal effects and the unit-specific and time-specific confounders. For rank-$1$ matrices, we use edge-disjoint paths to form an estimator that achieves minimax optimal estimation when the sampling is sufficiently dense. Our discovery introduces a new family of estimators parametrized by network flows, which provide a fine-grained and intuitive understanding of the impact of the given sampling pattern on the relative difficulty of estimation at an entry-specific level. This graph-based approach allows us to quantify the inherent complexity of matrix completion for individual entries, rather than relying solely on global measures of performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。