提出动态调度基准框架，揭示大模型调度代理的可观测性悖论。
DynaSchedBench: Calibrated Dynamic Scheduling Benchmarks and Observability Paradox in LLM-based Scheduling Agents
- 用新指标分层生成难度可控的调度实例，避免随机噪声干扰。
- 发现给大模型提供完整结构信息反而降低决策性能，存在可观测性悖论。
- 验证多数大模型调度器不如经典启发式算法，更像鲁棒近似器而非优化器。
当前基于神经组合优化的动态柔性作业车间调度问题（DFJSP）研究受限于方法论矛盾：静态基准易导致过拟合，而未校准的生成器则因随机噪声掩盖算法真实能力。为此，本文提出DynaSchedBench，一个针对DFJSP的诊断框架，通过严格控制实例生成过程来解决该问题。不同于参数采样，该框架采用顺序事件空间校准器（SESC），计算一种新型调度压力指数（SSI），按难度对实例进行分层。实验表明，SESC在计算效率上显著优于进化基线，且能可靠收敛至目标指标。该框架集成模块化组件，涵盖实例生成、基于快照的仿真、调度代理、评估与可视化，支持对反应式与前瞻策略的严格测试。利用此校准环境，我们揭示了基于大语言模型（LLM）调度代理的关键局限：在逐步在线决策中，赋予代理对完整结构信息的预言访问权反而会降低策略性能，其表现劣于简洁信息输入；尽管存在大量上下文开销，工具增强与精炼策略未能稳定提升性能，多数LLM代理无法持续超越强基线调度规则，表现出更像稳健启发式近似器而非优越优化器的行为。
原文摘要 · Abstract (English)
Progress in neural combinatorial optimization for Dynamic Flexible Job Shop Scheduling Problem (DFJSP) is currently hindered by a methodological tension: static benchmarks encourage benchmark overfitting, while uncalibrated generators obscure algorithmic capability with stochastic noise. To resolve this, we introduce \textbf{DynaSchedBench}, a diagnostic framework for DFJSP that rigorously controls the instance-generation process. Instead of relying on parameter sampling, our approach utilizes Sequential Event-Space Calibrator (SESC) that computes a novel Schedule Stress Index (SSI) to stratify instances by difficulty. We demonstrate that SESC is substantially more computationally efficient than evolutionary baselines while converging reliably to the target metrics. The framework integrates modular components for instance generation, snapshot-based simulation, agents, evaluation, and visualization, thereby enabling rigorous testing of reactive and lookahead-based policies. Leveraging this calibrated environment, we identify key limitations of LLM-based scheduling agents. Specifically, in step-wise online decision-making for dynamic scheduling, we identify an ``Observability Paradox'': providing agents with oracle access to full structural information can degrade policy performance, underperforming concise information. Furthermore, despite substantial token overhead, tool-augmented and refinement strategies fail to reliably improve performance, and most LLM agents fail to consistently surpass strong dispatching baselines-behaving more like robust heuristic approximators than superior optimizers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。