提出新评估框架,精准测试节奏游戏谱面生成质量
ChartGenEval: Corruption-Tested Multi-Dimensional Feedback for Rhythm-Game Chart Generation
- 用动态干扰测试替代固定指标,评估谱面生成的鲁棒性
- 在80组歌曲中验证7个输出维度满足敏感性与不变性要求
- 适合优化生成器或做任务定制化压力测试的开发者
生成的节奏游戏谱面无需完全复现官方音符序列:同一首歌和难度下存在多种合理音符选择。因此,参考音符匹配仅衡量重建程度,而非完整设计问题。我们提出ChartGenEval,一个包含六问的评估框架,其核心具备自动化的抗干扰测试能力。该框架开放音符选择,仅以官方谱面的时间标记作为锚点,不提供目标音符。通过剂量控制的故障注入测试,取代对常见统计量的盲目依赖。在80组保留歌曲中,七个输出轴在九项非冗余测试中均满足预设的敏感性与不变性标准。在40首歌曲的开发集上进行附加压力测试,揭示两项普遍规律:全局相位估计可恢复15、30、60毫秒的偏移,而仅依赖谱面的输出几乎不变;通用模式重写使语言模型困惑度降低37%,循环坍缩使自相似性提升62%。因此,ChartGenEval不提供单一代理评分,而是报告独立的角色特定信号。这一谱系化反馈可自动用于比较和迭代生成器,经任务特异性压力测试后的输出可作为优化目标或约束条件。
原文摘要 · Abstract (English)
A generated rhythm-game chart need not reproduce one official note sequence: many note choices can fit the same song and difficulty. Reference-note agreement therefore measures reconstruction, not the full design problem. We introduce ChartGenEval, a six-question evaluation framework with an automatic, corruption-tested core. It leaves note choice open while anchoring timing to the song: the matched official chart supplies only its authored timing map, never target notes. We test each core output with dose-controlled failures rather than assume that a familiar statistic measures chart quality. Across 80 held-out song groups, seven output axes satisfy prespecified sensitivity and invariance criteria in nine nonredundant tests. Complementary stress tests on the 40-song development panel expose two broader lessons. A chart-wide phase estimate recovers injected shifts of 15, 30, and 60 ms while chart-only outputs remain essentially unchanged. Common-pattern rewriting lowers mean language-model perplexity by 37%, and loop collapse raises mean self-similarity by 62%. ChartGenEval therefore reports separate, role-specific signals instead of one proxy or total score. This profile provides automatic feedback for comparing and iterating generators; selected outputs are candidate optimization targets or constraints after task-specific stress testing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。