无需信道信息，用分层强化学习控制可重构表面，提升毫米波网络信号覆盖。
Learning to Focus: CSI-Free Hierarchical MARL for Reconfigurable Reflectors
- 用用户定位数据替代信道估计，实现无信道状态信息的智能控制
- 分层架构使信号增益最高提升7.79 dB，优于集中式方案
- 适合大规模部署，对定位误差有强鲁棒性，适合实际无线环境
可重构智能表面（RIS）有望为下一代毫米波（mmWave）网络构建智能射频环境。然而，信道状态信息（CSI）估计带来的高昂计算开销，以及集中式优化固有的维度爆炸问题，严重阻碍了其大规模实用化。为此，我们提出一种基于分层多智能体强化学习（HMARL）的“无CSI”范式，用于控制机械式可重构反射表面。通过将基于导频的信道估计替换为可获取的用户定位数据，该框架利用空间智能实现宏观尺度的波传播管理。控制问题被分解为两级神经架构：高层控制器执行时间上延展的离散用户-反射面分配，低层控制器在集中训练、分散执行（CTDE）框架下，使用多智能体近端策略优化（MAPPO）自主优化连续焦点位置。基于确定性射线追踪的全面评估表明，该分层框架相比集中式基线，信号强度提升最高达7.79 dB。此外，系统展现出良好的多用户可扩展性，并在实际亚米级定位追踪误差下保持高度稳健的波束聚焦性能。通过消除CSI开销同时维持高保真信号重定向，本工作为智能无线环境提供了一种可扩展且低成本的蓝图。
原文摘要 · Abstract (English)
Reconfigurable Intelligent Surfaces (RIS) has a potential to engineer smart radio environments for next-generation millimeter-wave (mmWave) networks. However, the prohibitive computational overhead of Channel State Information (CSI) estimation and the dimensionality explosion inherent in centralized optimization severely hinder practical large-scale deployments. To overcome these bottlenecks, we introduce a ``CSI-free" paradigm powered by a Hierarchical Multi-Agent Reinforcement Learning (HMARL) architecture to control mechanically reconfigurable reflective surfaces. By substituting pilot-based channel estimation with accessible user localization data, our framework leverages spatial intelligence for macro-scale wave propagation management. The control problem is decomposed into a two-tier neural architecture: a high-level controller executes temporally extended, discrete user-to-reflector allocations, while low-level controllers autonomously optimize continuous focal points utilizing Multi-Agent Proximal Policy Optimization (MAPPO) under a Centralized Training with Decentralized Execution (CTDE) scheme. Comprehensive deterministic ray-tracing evaluations demonstrate that this hierarchical framework achieves massive RSSI improvements of up to 7.79 dB over centralized baselines. Furthermore, the system exhibits robust multi-user scalability and maintains highly resilient beam-focusing performance under practical sub-meter localization tracking errors. By eliminating CSI overhead while maintaining high-fidelity signal redirection, this work establishes a scalable and cost-effective blueprint for intelligent wireless environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。