用切片沃瑟斯坦距离筛选训练数据，提升关键场景下模型可靠性。
Sliced-Wasserstein Distance-based Data Selection
- 基于切片沃瑟斯坦距离设计无监督异常检测，实现保守数据选择。
- 提出两种高效近似方法，支持大规模数据处理，计算开销低。
- 适用于电力系统等高风险领域，可作为数据清洗与建模前的预处理工具。
我们提出一种基于切片沃瑟斯坦距离的新型无监督异常检测方法，用于机器学习中的训练数据筛选。该方法在电力系统等关键领域部署机器学习模型时具有重要意义，因其能实现保守的数据选择并具备最优传输的解释性。为保证方法可扩展性，我们提供了两种高效近似：第一种对数据集的低基数表示并行处理；第二种采用计算轻量的欧氏距离近似。此外，我们首次公开了一个展示北方气候下局部关键峰值返利需求响应的数据集。在合成数据集上展示了该方法的过滤模式，并在训练数据选择任务中进行了数值基准测试。最后，我们将该方法应用于我们开源数据集的首个预测基准测试中。
原文摘要 · Abstract (English)
We propose a new unsupervised anomaly detection method based on the sliced-Wasserstein distance for training data selection in machine learning approaches. Our filtering technique is interesting for decision-making pipelines deploying machine learning models in critical sectors, e.g., power systems, as it offers a conservative data selection and an optimal transport interpretation. To ensure the scalability of our method, we provide two efficient approximations. The first approximation processes reduced-cardinality representations of the datasets concurrently. The second makes use of a computationally light Euclidian distance approximation. Additionally, we open the first dataset showcasing localized critical peak rebate demand response in a northern climate. We present the filtering patterns of our method on synthetic datasets and numerically benchmark our method for training data selection. Finally, we employ our method as part of a first forecasting benchmark for our open-source dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。