对比三种强化学习算法在网页搜索匹配中的表现，发现新方法训练更快、效果更好。
Comparative Analysis of Parameterized Action Actor-Critic Reinforcement Learning Algorithms for Web Search Match Plan Generation
- 提出参数化动作的贪心策略，避免使用循环网络，简化决策过程
- 新算法在两个基准测试中最快完成5000轮训练，返回值最高
- 适合需要快速收敛和稳定性能的复杂动作空间任务
本研究在全可观测环境下评估了软演员-评论家（SAC）、贪心演员-评论家（GAC）和截断分位数评论家（TQC）在高维决策任务中的表现。聚焦于参数化动作（PA）空间，无需使用循环网络，并通过Platform-v0和Goal-v0基准测试离散动作与连续动作参数空间的关联性。利用Microsoft NNI进行超参数优化，对GAC和TQC代码库进行修改以确保可复现性。结果表明，参数化动作贪心演员-评论家（PAGAC）优于其他算法，在平台游戏和机器人足球进球游戏中分别仅用41分24秒和24分04秒完成5000个训练回合，且收益最高。其速度与稳定性优势显著，相比PASAC和PATQC更高效可靠，适用于要求快速收敛与强鲁棒性的任务。未来可探索熵正则化与截断方法结合的混合策略，以增强稳定性并扩展泛化性研究。
原文摘要 · Abstract (English)
This study evaluates the performance of Soft Actor Critic (SAC), Greedy Actor Critic (GAC), and Truncated Quantile Critics (TQC) in high-dimensional decision-making tasks using fully observable environments. The focus is on parametrized action (PA) spaces, eliminating the need for recurrent networks, with benchmarks Platform-v0 and Goal-v0 testing discrete actions linked to continuous action-parameter spaces. Hyperparameter optimization was performed with Microsoft NNI, ensuring reproducibility by modifying the codebase for GAC and TQC. Results show that Parameterized Action Greedy Actor-Critic (PAGAC) outperformed other algorithms, achieving the fastest training times and highest returns across benchmarks, completing 5,000 episodes in 41:24 for the Platform game and 24:04 for the Robot Soccer Goal game. Its speed and stability provide clear advantages in complex action spaces. Compared to PASAC and PATQC, PAGAC demonstrated superior efficiency and reliability, making it ideal for tasks requiring rapid convergence and robust performance. Future work could explore hybrid strategies combining entropy-regularization with truncation-based methods to enhance stability and expand investigations into generalizability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。