VPOS通过正交降维实现高效特征选择,保留原始变量可解释性。
Variance-Preserving Orthogonal Selection (VPOS): Greedy Feature Selection via Orthogonal Deflation in PCA Loading Space
- 在加权PCA载荷空间中逐次正交剔除,确保选中特征方向互补
- 在7个数据集上重建误差降低1%~78%,优于匹配的PCA方法
- 适合高维相关数据,兼具速度与可解释性,尤其适用于小样本场景
我们提出方差保持正交选择(VPOS),一种无监督特征选择方法,通过在方差加权主成分分析(PCA)载荷空间 $\mathbf{V}_d\mathbf{Λ}_d^{1/2}$ 中进行序列正交降维实现。每次选取特征后,将其载荷方向从剩余候选中投影剔除,使后续选择覆盖秩-$d$协方差近似的互补方向,同时恢复原始变量。我们建立了秩缩减保证和行列式增长解释,区分了VPOS与原始数据贪婪选择、未加权特征向量轮换、主特征分析(PFA)及主变量选择(PVS)。实验设定 $k\leq d$,在验证集上调参并评估未见外层折叠结果。在七个标注基准上,VPOS在所有数据集上均优于不带降维的匹配PCA,重建误差降低1%–78%;在Wine、乳腺癌和MNIST上取得最低平均重建误差,在CIFAR-10和HighDim上误差仅比最优值高1.7%。在CIFAR-10上,VPOS比相近的PVS基线快约24倍,重建误差差距仅为1.7%。这些结果确立了VPOS作为高效协方差覆盖方法,特别适用于需用少量可识别原始变量表示相关高维数据的场景。
原文摘要 · Abstract (English)
We present Variance-Preserving Orthogonal Selection (VPOS), an unsupervised feature-selection method that performs sequential orthogonal deflation in the variance-weighted principal component analysis (PCA) loading space $\mathbf{V}_d\mathbfΛ_d^{1/2}$. After each feature is selected, its loading direction is projected out of all remaining candidates, so subsequent selections cover complementary directions of the rank-$d$ covariance approximation while returning original variables. We establish rank-reduction guarantees and a determinant-growth interpretation, and distinguish VPOS from greedy selection on raw data, unweighted eigenvector pivoting, Principal Feature Analysis (PFA), and Principal Variable Selection (PVS). Experiments enforce $k\leq d$, tune method-specific parameters on validation observations, and evaluate on unseen outer folds. Across seven labelled benchmarks, VPOS improves held-out normalised reconstruction error over matched PCA without deflation on every dataset, with reductions of 1--78%. It obtains the lowest mean reconstruction error on Wine, Breast Cancer, and MNIST and is within 1.7% of the lowest error on CIFAR-10 and HighDim. On CIFAR-10, VPOS is approximately 24$\times$ faster than the closely related PVS baseline while incurring a 1.7% reconstruction gap. These results establish VPOS as an efficient covariance-coverage method, particularly when correlated high-dimensional data must be represented by a small set of identifiable original variables.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。