研究如何分组数据以实现最优且私密的机器学习，提升回归任务性能。
Aggregating Data for Optimal and Private Learning
- 通过聚类特征或标签优化数据分组策略，提升下游线性回归效果。
- 理论证明最优分组等价于k-means等经典聚类目标，可量化性能保证。
- 方法支持标签差分隐私，适用于对隐私敏感的数据场景。
多实例回归（MIR）与基于标签比例的学习（LLP）是许多应用中常见的学习框架，其中训练数据被划分为互不重叠的集合（袋子），学习者仅能获取每个袋子的聚合标签：MIR中为袋内某未公开实例的标签，LLP中为袋内标签的均值。本文研究在多种损失函数下，如何最优地将数据集划分为袋子，以最大化下游任务（如线性回归）的效用。我们从理论上提供了效用保障，并证明在每种情形下，最优分组策略（近似）等价于对特征向量或标签进行最优聚类，目标为自然的$ k $-means。同时，我们证明该分组机制可实现标签差分隐私，仅引入额外效用误差。最后，我们将结果推广至广义线性模型（GLMs），并通过实验验证了理论结论。
原文摘要 · Abstract (English)
Multiple Instance Regression (MIR) and Learning from Label Proportions (LLP) are learning frameworks arising in many applications, where the training data is partitioned into disjoint sets or bags, and only an aggregate label i.e., bag-label for each bag is available to the learner. In the case of MIR, the bag-label is the label of an undisclosed instance from the bag, while in LLP, the bag-label is the mean of the bag's labels. In this paper, we study for various loss functions in MIR and LLP, what is the optimal way to partition the dataset into bags such that the utility for downstream tasks like linear regression is maximized. We theoretically provide utility guarantees, and show that in each case, the optimal bagging strategy (approximately) reduces to finding an optimal clustering of the feature vectors or the labels with respect to natural objectives such as $k$-means. We also show that our bagging mechanisms can be made label-differentially private, incurring an additional utility error. We then generalize our results to the setting of Generalized Linear Models (GLMs). Finally, we experimentally validate our theoretical results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。