首个面向主动特征获取的标准化评估框架,解决方法对比无统一标准问题。
AFABench: A Generic Framework for Benchmarking Active Feature Acquisition
- 构建通用框架,支持多种特征获取策略的公平对比。
- 引入新数据集CUBE-NM,揭示贪婪策略在长远规划上的局限性。
- 适合研究低成本机器学习、智能数据采集的学者使用。
在许多实际场景中,获取数据实例的所有特征可能因成本、延迟或隐私问题而不可行。主动特征获取(AFA)通过动态为每个数据实例选择一组有信息量的特征,以预测性能与获取成本之间的权衡来应对这一挑战。尽管已有众多AFA方法被提出,涵盖从短视的信息论策略到非短视的强化学习方法,但缺乏标准化基准严重阻碍了这些方法的系统性评估。本文提出AFABench,首个AFA基准框架。该框架包含多样化的合成与真实世界数据集,支持广泛特征获取策略,并采用模块化设计,便于集成新方法与任务。我们实现了并评估了来自各主要类别(静态、短视、强化学习)的代表性算法。为测试AFA策略的前瞻能力,我们引入新型合成数据集CUBE-NM,专门暴露短视选择的局限性。结果揭示了不同AFA策略的关键权衡,并为未来研究提供可操作洞见。基准代码已开源:https://github.com/Linusaronsson/AFA-Benchmark。
原文摘要 · Abstract (English)
In many real-world scenarios, acquiring all features of a data instance can be expensive or impractical due to monetary cost, latency, or privacy concerns. Active Feature Acquisition (AFA) addresses this challenge by dynamically selecting a subset of informative features for each data instance, trading predictive performance against acquisition cost. While numerous methods have been proposed for AFA, ranging from myopic information-theoretic strategies to non-myopic reinforcement learning approaches, fair and systematic evaluation of these methods has been hindered by a lack of standardized benchmarks. In this paper, we introduce AFABench, the first benchmark framework for AFA. Our benchmark includes a diverse set of synthetic and real-world datasets, supports a wide range of acquisition policies, and provides a modular design that enables easy integration of new methods and tasks. We implement and evaluate representative algorithms from all major categories, including static, myopic, and reinforcement learning-based approaches. To test the lookahead capabilities of AFA policies, we introduce a novel synthetic dataset, CUBE-NM, designed to expose the limitations of myopic selection. Our results highlight key trade-offs between different AFA strategies and provide actionable insights for future research. The benchmark code is available at: https://github.com/Linusaronsson/AFA-Benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。