提出可让普通人描述强化学习问题的通用语言，提升易用性和泛化能力
A Research Agenda for Usability and Generalisation in Reinforcement Learning
- 设计用户友好的问题描述语言，降低编程门槛
- 使算法能基于统一格式快速适应新任务
- 适合无工程背景的研究者和小团队使用
强化学习研究中，智能体通常在定制化的模拟器中训练与部署，依赖工程师使用通用编程语言或硬件加速框架（如CUDA、JAX）实现。这不仅要求开发算法需具备编程能力，也要求使用者具备相同技能来解决新问题，导致强化学习难以普及。我们还观察到，缺乏统一的问题形式化表达标准，使得不同任务间无法建立一致表示，阻碍了智能体对新问题的快速泛化。本文提出一项研究议程：构建用户友好的问题描述语言，使无工程背景的用户也能正式描述待解决的问题，并让算法利用这些描述实现跨任务的有效泛化。
原文摘要 · Abstract (English)
It is common practice in reinforcement learning (RL) research to train and deploy agents in bespoke simulators, typically implemented by engineers directly in general-purpose programming languages or hardware acceleration frameworks such as CUDA or JAX. This means that programming and engineering expertise is not only required to develop RL algorithms, but is also required to use already developed algorithms for novel problems. The latter poses a problem in terms of the usability of RL, in particular for private individuals and small organisations without substantial engineering expertise. We also perceive this as a challenge for effective generalisation in RL, in the sense that is no standard, shared formalism in which different problems are represented. As we typically have no consistent representation through which to provide information about any novel problem to an agent, our agents also cannot instantly or rapidly generalise to novel problems. In this position paper, we advocate for a research agenda centred around the use of user-friendly description languages for describing problems, such that (i) users with little to no engineering expertise can formally describe the problems they would like to be tackled by RL algorithms, and (ii) algorithms can leverage problem descriptions to effectively generalise among all problems describable in the language of choice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。