arXiv:2605.03660cs.MMcs.AI2026-05

用模仿学习实现多灯光的音乐驱动自动控制,灵活适配不同演出场景。

Stage Light is Sequence$^2$: Multi-Light Control via Imitation Learning

论文配图:Stage Light is Sequence$^2$: Multi-Light Control via Imitation Learning
图 1 · 摘自论文原文
  • 分层框架将音乐映射到多灯HSV色彩空间,先生成全局色分布再分解至各灯。
  • 仅需混合光数据即可训练,无需专业操作示范,在不同场地表现良好。
  • 提出基于目标条件的马尔可夫决策过程与三阶段模仿学习,提升泛化能力。

近年来,音乐驱动的自动舞台灯光控制(ASLC)因专业灯光师人力与时间成本高昂而受到关注。现有方法存在规则不可解释、仅支持单主灯控制、音乐到参数映射迁移性差等局限。为此,我们提出SeqLight,一种将音乐映射至多灯色调-饱和度-明度(HSV)空间的分层深度学习框架。首先定制SkipBART模型生成每帧的全局灯光颜色分布,随后采用混合模仿学习(IL)策略,设计出有效的多灯颜色分配方案。关键在于,该分解模块可在不同场地配置下,仅使用混合灯光数据进行训练,无需专业操作示范,具备强适应性。在此阶段,我们将灯光分解任务建模为带目标条件的马尔可夫决策过程(GCMDP),构建受启发于后见经验重播(HER)的专家示范集,并设计三阶段模仿学习训练流程,实现优异泛化性能。通过定量分析与人工评测验证了所提方法的有效性。代码与训练模型已公开于https://github.com/RS2002/SeqLight。

原文摘要 · Abstract (English)

Music-inspired Automatic Stage Lighting Control (ASLC) has gained increasing attention in recent years due to the substantial time and financial costs associated with hiring and training professional lighting engineers. However, existing methods suffer from several notable limitations: the low interpretability of rule-based approaches, the restriction to single-primary-light control in music-to-color-space methods, and the limited transferability of music-to-controlling-parameter frameworks. To address these gaps, we propose SeqLight, a hierarchical deep learning framework that maps music to multi-light Hue-Saturation-Value (HSV) space. Our approach first customizes SkipBART, an end-to-end single primary light generation model, to predict the full light color distribution for each frame, followed by hybrid Imitation Learning (IL) techniques to derive an effective decomposition strategy that distributes the global color distribution among individual lights. Notably, the light decomposition module can be trained under varying venue-specific lighting configurations using only mixed light data and no professional demonstrations, thereby flexibly adapting across diverse venues. In this stage, we formulate the light decomposition task as a Goal-Conditioned Markov Decision Process (GCMDP), construct an expert demonstration set inspired by Hindsight Experience Replay (HER), and introduce a three-phase IL training pipeline, achieving strong generalization capability. To validate our IL solution for the proposed GCMDP, we conduct a series of quantitative analysis and human study. The code and trained models are provided at https://github.com/RS2002/SeqLight .

舞台灯光模仿学习多光源控制音乐生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。