让AI代理在执行任务时自动调整配置,实现动态适应。
ToolSelf: Unifying Task Execution and Self-Reconfiguration via Tool-Driven Emergent Adaptation
- 将配置修改封装为工具接口,与任务执行统一在同一个决策空间中。
- 训练后平均性能比固定配置模型高28.8分,零样本表现媲美专用代理。
- 适合需要自适应调整的复杂长程任务场景,如智能体系统开发。
基于大语言模型的智能体系统在处理复杂长周期任务时表现出色,但受限于执行前固定的静态配置。这种僵化导致领域专精与跨任务泛化之间的权衡:强先验和小工具集利于专业化却削弱迁移能力,而通用流程和宽泛动作空间虽扩大覆盖范围却稀释指导性。现有预执行优化、规划者-工作者编排及配置修补方法无法化解此矛盾,因它们将适应与执行分离,造成信息丢失、优化碎片化和责任归属模糊。本文提出ToolSelf,一种通过工具驱动实现运行时自我重构的范式,将配置更新抽象为标准化工具接口,并将执行与适应统一于单一策略的动作空间中。执行智能体可根据任务进展与反馈动态调整子目标、策略、工具箱、上下文及上下文管理模式。进一步提出配置感知两阶段训练(CAT),结合拒绝采样微调与轨迹级KTO强化学习,内化自我重构能力。在多个基准测试中,零样本下的ToolSelf表现媲美任务专用代理;经CAT训练后,平均性能较静态配置基线提升28.8分,揭示了一条无需人工注入指导即可实现涌现自适应的新路径。代码已开源。
原文摘要 · Abstract (English)
LLM-powered agentic systems excel at complex long-horizon tasks, but remain constrained by static configurations fixed before execution. Such rigidity forces a trade-off between domain-specific performance and cross-task generalization: strong priors and compact tool spaces aid specialization but weaken transfer, while task-agnostic workflows and broad action spaces expand coverage but dilute guidance. Existing pre-execution optimization, planner-worker orchestration, and configuration patching fall short of resolving this tension, as they decouple adaptation from execution, causing information loss, fragmented optimization, and ambiguous credit assignment. We propose ToolSelf, a tool-driven runtime self-reconfiguration paradigm that abstracts configuration updates as a standardized tool interface and unifies execution and adaptation within one policy's action space. The execution agent can dynamically update sub-goals, strategies, toolboxes, context, and context-management modes based on task progress and feedback. We further introduce Configuration-Aware Two-stage Training (CAT), which combines rejection sampling fine-tuning with trajectory-level KTO reinforcement learning to internalize self-reconfiguration. Across diverse benchmarks, zero-shot ToolSelf rivals task-specialized agents; after CAT training, ToolSelf gains 28.8 points over the static-configuration baseline on average, illuminating a path toward emergent adaptivity that obviates manually injected guidance. The code is available at https://github.com/lian-tian-mo-zun/ToolSelf.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。