研究提示词剧本在不同场景下的迁移效果，发现其效果受解码策略和环境差异影响。
When Do Prompt-Side Agent Playbooks Transfer? Accuracy, Cost, and Runtime Shift in Agent Deployment
- 采用统一的蒸馏-验证-迁移流程测试剧本跨场景复用性。
- 在特定条件下，蒸馏指导比固定示范更优，但多数效果不显著。
- 剧本迁移需目标环境验证，不能直接复用，适合资源受限的冷启动场景。
提示词侧剧本可在不重新训练的情况下提升工具使用型语言代理性能，但其在源设置外的可移植性尚不明确。本文在共享蒸馏-验证-迁移协议下研究冻结剧本的迁移效果。在ALFWorld上，控制贪婪解码时迁移有益；在一次近预算匹配比较中，蒸馏引导优于五个固定示范。在TAU2-Bench上，预设聚合对比显示适度的同域优势，但全局霍尔姆校正仅保留135个路线级效应中的一个；其余结果提供兼容性敏感异质性的描述性证据。在XBench-DeepSearch中，一种特定的运行时-剧本配对虽保留了首次尝试启发式，却导致重复查询、停止延迟及显著成本上升。跨基准分析表明，迁移与目标生成的剧本均需在目标端验证成功性、终止条件、协议兼容性和成本。因此，冻结迁移仅为条件性冷启动选项，而非默认复用策略或普遍优于目标端重蒸馏的替代方案。
原文摘要 · Abstract (English)
Prompt-side playbooks can improve tool-using language agents without retraining, but their portability beyond the source setting is unclear. We study frozen playbook transfer under a shared distill--validate--transfer protocol. On ALFWorld, transfer is beneficial under controlled greedy decoding and, in one near-budget-matched comparison, distilled guidance outperforms five fixed demonstrations. On TAU2-Bench, a prespecified aggregate contrast supports a modest average matched-domain advantage, but global Holm correction retains only one of 135 route-level effects; the remaining grid provides descriptive evidence of compatibility-sensitive heterogeneity. On XBench-DeepSearch, one artifact--runtime pairing preserves useful first-try heuristics while producing repeated queries, delayed stopping, and substantial cost inflation after a context-runtime shift. Across benchmarks, transferred and target-derived playbooks both require target-side validation of success, termination, protocol compatibility, and cost. Frozen transfer is therefore a conditional cold-start option, not a reuse-by-default strategy or a universally preferable alternative to target-side redistillation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。