评测大模型在复杂太空任务规划中的综合能力，发现通用模型表现远逊专业求解器。
AstroReason-Bench: Evaluating Unified Agentic Planning across Heterogeneous Space Planning Problems
- 构建统一交互协议，覆盖地面站通信与敏捷地球观测等多种任务场景。
- 在真实物理约束下，通用大模型性能显著低于专用求解器。
- 适合研究智能体规划、航天任务优化的学者和开发者参考。
近期基于智能体的大语言模型（LLMs）展现出跨多样化任务进行推理与决策的通用规划能力。然而，现有智能体评测多集中于符号化或弱约束环境，对其在具物理约束的真实世界领域中的表现关注不足。本文提出 AstroReason-Bench，一个面向空间规划问题（SPP）的综合性评测基准，涵盖具有异质目标、严格物理约束及长时程决策特点的高风险任务。该基准整合了地面站通信与敏捷地球观测等多种调度模式，并提供统一的智能体交互协议。对多种前沿开源与闭源智能体大模型进行评估，结果表明当前通用智能体在真实约束下表现显著落后于专用求解器，揭示了通用规划在现实场景中的关键局限。AstroReason-Bench 为未来智能体研究提供了挑战性强且诊断性高的测试平台。
原文摘要 · Abstract (English)
Recent advances in agentic Large Language Models (LLMs) have positioned them as generalist planners capable of reasoning and acting across diverse tasks. However, existing agent benchmarks largely focus on symbolic or weakly grounded environments, leaving their performance in physics-constrained real-world domains underexplored. We introduce AstroReason-Bench, a comprehensive benchmark for evaluating agentic planning in Space Planning Problems (SPP), a family of high-stakes problems with heterogeneous objectives, strict physical constraints, and long-horizon decision-making. AstroReason-Bench integrates multiple scheduling regimes, including ground station communication and agile Earth observation, and provides a unified agent-oriented interaction protocol. Evaluating on a range of state-of-the-art open- and closed-source agentic LLM systems, we find that current agents substantially underperform specialized solvers, highlighting key limitations of generalist planning under realistic constraints. AstroReason-Bench offers a challenging and diagnostic testbed for future agentic research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。