用数据驱动方法帮推荐系统研究选对数据集。
Informed Dataset Selection
- 基于算法性能空间框架，量化分析96个数据集的难易度。
- 在28个算法、5个K值下评估3项指标，识别最优数据集组合。
- 适合想科学选数据集的研究者，避免凭流行度选错数据。
推荐系统研究中的数据集选择缺乏系统方法，常依赖热度而非实际适用性。我们开发了APS Explorer网页工具，实现算法性能空间（APS）框架，分析96个数据集、28种算法，在nDCG、Hit Ratio、Recall三个指标下，于五个K值上进行评估。通过基于统计的分类系统，将数据集按五分位数划分为五个难度等级；引入基于马氏距离的方差归一化相似度度量。工具包含三个交互模块：算法性能可视化、算法直接对比、数据集元数据分析。该系统将数据集选择从直觉导向转变为证据驱动，现已公开访问：datasets.recommender-systems.com。
原文摘要 · Abstract (English)
The selection of datasets in recommender systems research lacks a systematic methodology. Researchers often select datasets based on popularity rather than empirical suitability. We developed the APS Explorer, a web application that implements the Algorithm Performance Space (APS) framework for informed dataset selection. The system analyzes 96 datasets using 28 algorithms across three metrics (nDCG, Hit Ratio, Recall) at five K-values. We extend the APS framework with a statistical based classification system that categorizes datasets into five difficulty levels based on quintiles. We also introduce a variance-normalized distance metric based on Mahalanobis distance to measure similarity. The APS Explorer was successfully developed with three interactive modules for visualizing algorithm performance, direct comparing algorithms, and analyzing dataset metadata. This tool shifts the process of selecting datasets from intuition-based to evidence-based practices, and it is publicly available at datasets.recommender-systems.com.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。