arXiv:2501.16863cs.LG2025-01被引 5

用高维计算解决上下文赌博机问题,更省资源、更快收敛。

HD-CB: The First Exploration of Hyperdimensional Computing for Contextual Bandits Problems

  • 将状态和动作映射为高维向量,用向量运算替代传统复杂计算
  • 在合成与真实数据上表现优于或媲美经典算法,收敛更快
  • 适合低算力设备,支持并行计算,参数少、内存开销小

高维计算(HDC),又称向量符号架构,是一种结合符号推理与分布式连接模型优势的计算范式。因其能效高、可扩展性强、抗噪声和硬件故障,近年来被视为资源受限环境下学习任务的有前景替代方案。本文首次探索将HDC应用于上下文赌博机(CB)问题,提出超维上下文赌博机(HD-CB):将环境状态映射到高维空间,每个动作用专用超向量(HV)表示。每轮迭代中,利用这些超向量选择最优动作,并根据反馈奖励更新,取代传统线性CB算法中的高复杂度岭回归,转而采用简单且高度并行的向量操作。我们设计了四种HD-CB变体,实现不同探索策略,同时提出降低内存开销与超参数数量的技术。在合成数据集与真实世界基准上的大量模拟实验表明,HD-CB在性能上持续达到或超越传统线性CB算法,具备更快速的收敛速度、更低的计算复杂度、更好的可扩展性与高并行性。

原文摘要 · Abstract (English)

Hyperdimensional Computing (HDC), also known as Vector Symbolic Architectures, is a computing paradigm that combines the strengths of symbolic reasoning with the efficiency and scalability of distributed connectionist models in artificial intelligence. HDC has recently emerged as a promising alternative for performing learning tasks in resource-constrained environments thanks to its energy and computational efficiency, inherent parallelism, and resilience to noise and hardware faults. This work introduces the Hyperdimensional Contextual Bandits (HD-CB): the first exploration of HDC to model and automate sequential decision-making Contextual Bandits (CB) problems. The proposed approach maps environmental states in a high-dimensional space and represents each action with dedicated hypervectors (HVs). At each iteration, these HVs are used to select the optimal action for the given context and are updated based on the received reward, replacing computationally expensive ridge regression procedures required by traditional linear CB algorithms with simple, highly parallel vector operations. We propose four HD-CB variants, demonstrating their flexibility in implementing different exploration strategies, as well as techniques to reduce memory overhead and the number of hyperparameters. Extensive simulations on synthetic datasets and a real-world benchmark reveal that HD-CB consistently achieves competitive or superior performance compared to traditional linear CB algorithms, while offering faster convergence time, lower computational complexity, improved scalability, and high parallelism.

高维计算上下文赌博机高效算法低功耗

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。