提出一种更高效且可扩展的主动学习算法，大幅降低存储与计算开销。
A Scalable Algorithm for Active Learning
- 采用近似计算策略减少存储与计算复杂度
- 在多个数据集上保持原算法精度，支持三百万点大规模数据
- 支持多GPU并行，适合大规模主动学习场景
FIRAL是一种基于逻辑回归的多分类确定性主动学习算法，已被证明在准确率和鲁棒性上优于现有方法，并具备理论性能保证。然而，当面对大规模数据集（点数n、维度d、类别数c较大）时，其存储需求为$\mathcal{O}(c^2d^2+nc^2d)$，计算复杂度高达$\mathcal{O}(c^3(nd^2 + bd^3 + bn))$，其中$b$为每次选择的样本数，导致可扩展性受限。为此，本文提出一种近似算法，将存储需求降至$\mathcal{O}(n(d+c) + cd^2)$，计算复杂度优化至$\mathcal{O}(bncd^2)$。同时提供GPU并行实现。在MNIST、CIFAR-10、Caltech101和ImageNet上的实验表明，该方法在精度上与FIRAL无显著差异；在最多12个GPU上对三百万点合成数据进行强弱扩展测试，验证了其优异的可扩展性。
原文摘要 · Abstract (English)
FIRAL is a recently proposed deterministic active learning algorithm for multiclass classification using logistic regression. It was shown to outperform the state-of-the-art in terms of accuracy and robustness and comes with theoretical performance guarantees. However, its scalability suffers when dealing with datasets featuring a large number of points $n$, dimensions $d$, and classes $c$, due to its $\mathcal{O}(c^2d^2+nc^2d)$ storage and $\mathcal{O}(c^3(nd^2 + bd^3 + bn))$ computational complexity where $b$ is the number of points to select in active learning. To address these challenges, we propose an approximate algorithm with storage requirements reduced to $\mathcal{O}(n(d+c) + cd^2)$ and a computational complexity of $\mathcal{O}(bncd^2)$. Additionally, we present a parallel implementation on GPUs. We demonstrate the accuracy and scalability of our approach using MNIST, CIFAR-10, Caltech101, and ImageNet. The accuracy tests reveal no deterioration in accuracy compared to FIRAL. We report strong and weak scaling tests on up to 12 GPUs, for three million point synthetic dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。