通过智能重置点提升电路优化样本效率
Lighthouse RL: Sample-Efficient Circuit Optimization via Strategic Reset Points
- 从训练中发现的高性能状态重启，引导探索到有希望区域
- 样本效率提升1.72倍，成功率达100%（对比0-87%）
- 适用于昂贵黑箱优化，可直接嵌入现有强化学习框架
本文提出Lighthouse RL，一种面向模拟电路尺寸优化的高效强化学习方法。传统方法在不同性能目标间泛化能力差，标准RL则浪费资源在无望区域探索。本方法通过战略重置策略，从训练中发现的高性能配置（称作'灯塔'）重启，这些状态更接近目标，能有效引导探索。在二维基准问题和两个模拟电路上的实验表明，相比文献中的RL与贝叶斯优化方法，本方法显著提升样本效率（最快快1.72倍）、优化成功率（100%对0-87%）、泛化能力（75%对0-50%外推成功率）及目标最大化效果。该策略特别适用于计算成本高的黑箱优化问题，且可作为即插即用组件集成到任意基于RL的优化系统中。
原文摘要 · Abstract (English)
In this paper, we introduce Lighthouse RL, a sample-efficient reinforcement learning (RL) approach for analog circuit sizing. Traditional methods lack generalization across different performance targets, while standard RL approaches waste resources exploring unpromising regions. Our method addresses these inefficiencies through a strategic reset strategy that initializes episodes from high-performing configurations discovered during training, called "lighthouses". These states, which are closer to the target objectives, guide exploration toward promising regions. When compared to RL and Bayesian optimization methods from the literature, we demonstrate the effectiveness of our approach on a 2D benchmark problem and on two analog circuits, showing significant improvements in sample efficiency (up to 1.72x faster), optimization performance (100% vs. 0-87% success rate), generalization (75% vs. 0-50% extrapolation success), and objective maximization. This efficiency is particularly valuable for computationally expensive black-box optimization problems, and our reset strategy can be used as a plug-and-play enhancement for any RL-based optimization approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。