arXiv:2503.02238cs.CL2025-03被引 5

测试大模型在限时任务中并行执行的规划能力,发现效率与可行性难兼顾。

Haste Makes Waste: Evaluating Planning Abilities of LLMs for Efficient and Feasible Multitasking with Time Constraints Between Actions

  • 基于真实烹饪场景设计多任务限时并行基准
  • 现有模型在追求效率时常违反时间约束导致失败
  • 适合研究多任务规划、时序推理的AI开发者

尽管基于大语言模型的智能体在任务完成上取得显著进展,现有评估基准往往过度强调单任务表现,忽视了现实场景中多任务规划与执行效率的关键需求。为填补这一空白,我们提出Recipe2Plan——一个基于真实烹饪场景的新基准框架。与传统基准不同,Recipe2Plan要求智能体通过并行任务执行优化烹饪时间,同时严格遵守动作间的时序约束:特定操作必须在前序步骤后特定时间窗口内完成。过于激进的局部并行可能破坏该约束,从而影响整个烹饪流程。这种严格的动作间时间约束对智能体提出了独特挑战:需在最大化并发操作与遵守关键时序约束之间取得平衡。对先进模型的大量实验表明,维持这一平衡存在显著困难。结果凸显了大语言模型在时序感知与全局多任务能力上的不足。我们已将基准与代码开源至https://github.com/WilliamZR/Recipe2Plan。

原文摘要 · Abstract (English)

While Large Language Model-based agents have demonstrated substantial progress in task completion, existing evaluation benchmarks tend to overemphasize single-task performance, with insufficient attention given to the crucial aspects of multitask planning and execution efficiency required in real-world scenarios. To bridge this gap, we present Recipe2Plan, a novel benchmark framework based on real-world cooking scenarios. Unlike conventional benchmarks, Recipe2Plan challenges agents to optimize cooking time through parallel task execution while respecting temporal constraints i.e. specific actions need to be performed within a particular time intervals following the preceding steps. Overly aggressive local parallelization may disrupt this constraint, potentially compromising the entire cooking process. This strict time constraint between actions raises a unique challenge for agents to balance between maximizing concurrent operations and adhering to critical timing constraints. Extensive experiments with state-of-the-art models reveal challenges in maintaining this balance between efficiency and feasibility. The results highlight the need for improved temporal awareness and global multitasking capabilities in large language models. We open-source our benchmark and code at https://github.com/WilliamZR/Recipe2Plan.

多任务规划时序约束大模型评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。