构建动态传感环境下的图主动学习评估基准,解决现有方法忽视用户负担的问题。
GRAIL: A Benchmark for GRaph ActIve Learning in Dynamic Sensing Environments
- 提出基于图结构的主动学习策略,通过节点重要性选择减少标注需求。
- 实测显示预测性能与用户负担存在权衡,传统方法在动态场景下表现不佳。
- 适合研究动态环境下智能感知、人机交互及低负担数据采集的学者。
基于图的主动学习(AL)利用图结构高效优先选择需标注的节点,降低健康监测、行为分析和传感器网络中的标注成本与用户负担。通过识别关键节点,图AL在保持模型性能的同时减少数据采集量,适用于动态环境。然而,现有方法多在静态图数据集上评估,侧重预测准确率,忽略采样多样性、查询公平性及对动态变化的适应性等用户中心考量。为此,本文提出GRAIL——一个面向动态真实传感环境的图主动学习评估基准框架。GRAIL引入新指标,用于评估持续有效性、采样多样性和用户负担,实现对不同条件下主动学习策略的全面评估。在包含动态真实人类传感数据的多个数据集上的大量实验揭示了预测性能与用户负担间的权衡,暴露了现有方法的局限性。结果表明,在动态环境中需平衡节点重要性、查询多样性与网络拓扑特征,GRAIL为图主动学习解决方案提供了有效的评估机制。
原文摘要 · Abstract (English)
Graph-based Active Learning (AL) leverages the structure of graphs to efficiently prioritize label queries, reducing labeling costs and user burden in applications like health monitoring, human behavior analysis, and sensor networks. By identifying strategically positioned nodes, graph AL minimizes data collection demands while maintaining model performance, making it a valuable tool for dynamic environments. Despite its potential, existing graph AL methods are often evaluated on static graph datasets and primarily focus on prediction accuracy, neglecting user-centric considerations such as sampling diversity, query fairness, and adaptability to dynamic settings. To bridge this gap, we introduce GRAIL, a novel benchmarking framework designed to evaluate graph AL strategies in dynamic, real-world environments. GRAIL introduces novel metrics to assess sustained effectiveness, diversity, and user burden, enabling a comprehensive evaluation of AL methods under varying conditions. Extensive experiments on datasets featuring dynamic, real-life human sensor data reveal trade-offs between prediction performance and user burden, highlighting limitations in existing AL strategies. GRAIL demonstrates the importance of balancing node importance, query diversity, and network topology, providing an evaluation mechanism for graph AL solutions in dynamic environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。