CIPHER通过生成与选择分离,提升数据科学代理在测试时的鲁棒性。
CIPHER: A Decoupled Exploration-Selection Framework for Test-Time Scaling of Data Science Agents

- 将初始状态生成与选择解耦,支持并行探索多个起点。
- 在两个基准上超越现有模型,且用小模型达大模型性能。
- 揭示了生成、选择策略对效果的影响,指导实践设计。
数据科学任务涵盖从封闭式信息提取到开放式分析的广泛范围,自动化面临巨大挑战。近年来,基于语言模型的AI代理在处理此类复杂任务方面展现出潜力。然而,现有代理通常依赖单一初始状态来驱动整个执行过程,容易因初始状态不佳引发级联错误。为缓解此问题,我们提出CIPHER——一种通过生成并选择多个初始状态进行并发执行的自动化数据科学代理,实现测试时扩展。不同于以往测试时扩展工作,CIPHER明确将候选初始状态的生成与战略选择解耦,用于并行执行。在两个基准(封闭式和开放式任务)上的广泛评估表明,CIPHER在匹配模型比较中超越当前最优表现,并在使用显著更小的基础语言模型情况下,仍保持与更大模型基线相当的竞争力。我们的实证研究刻画了解耦探索-选择(DES)框架的设计空间:量化了生成策略、选择策略及聚合模型容量对整体性能的贡献,并为从业者提供了可操作的设计建议。
原文摘要 · Abstract (English)
Data science tasks span from closed-ended information extraction to open-ended analysis, presenting significant challenges for automation. Recent AI agents powered by language models show promise for handling such complex tasks. However, existing agents typically rely on a single initial state that conditions the entire agent's execution, making them vulnerable to cascading errors initiated by a suboptimal initial state. To mitigate this, we present CIPHER, an automated data science agent that leverages test-time scaling through the generation and selection of multiple initial states for concurrent execution. Unlike existing works on test-time scaling of AI agents, CIPHER explicitly decouples the generation of candidate initial states from their strategic selection for parallel execution. Through extensive evaluation on two benchmarks (closed-form and open-form tasks), we demonstrate that CIPHER exceeds state-of-the-art performance in matched-model comparisons, and remains competitive against larger-model baselines despite relying on a substantially smaller base LM. Our empirical study characterizes the design space of the Decoupled Exploration-Selection (DES) framework: we quantify how generation strategy, selection strategy, and aggregator model capacity contribute to overall performance, and derive actionable design recommendations for practitioners.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。