arXiv:2605.24900cs.AI2026-05ACL

让智能助手提前准确触发任务,提升响应时机与动作一致性。

ProActor: Timing-Aware Reinforcement Learning for Proactive Task Scheduling Agents

论文配图:ProActor: Timing-Aware Reinforcement Learning for Proactive Task Scheduling Agents
图 1 · 摘自论文原文
  • 用动态时间窗口替代固定标签,实现可扩展的主动行为强化学习。
  • 在两个新数据集上显著提升触发时机准确性,动作一致性强于现有模型。
  • 适合研究主动式对话系统、智能助手优化的开发者与研究员。

主动任务导向代理需自主预判用户需求,识别可行动机会,并在恰当时刻触发软件操作,从根本上突破传统被动等待指令的局限。然而,现有方法缺乏通用的端到端方案来衡量与优化此类前瞻性行为。本文提出 ProActor,一个统一的对话任务调度框架,包含:(1) 无需领域依赖的自动化标注方法,通过生成完整机会时间窗口而非固定点标签,支持可扩展的主动行为强化学习;(2) 系统性主动度指标,同时捕捉触发时机质量与参考动作对齐度;(3) 基于 GRPO 的强化学习优化,结合多种奖励设计。研究发现,基于 RULER 的奖励与主动度评估标准对提升时机准确性至关重要,而阶段感知的复合奖励设计是平衡时机质量与动作对齐的关键。时序感知的强化学习需大量探索,为此我们构建 ART-F 框架,结合请求自适应推理集群与单机多卡 DDP 训练,实现 4 位量化 Qwen2.5-14B-ProActor-Q4 模型的高效 LoRA 训练,速度提升 4–8 倍。在两个新自标注数据集上的实验表明,该方法在显著改善主动触发时机的同时,保持了与最先进基线相当的动作一致性。消融实验验证了不同复合奖励设计的有效性。

原文摘要 · Abstract (English)

Proactive task-oriented agents must autonomously anticipate user needs, identify actionable opportunities, and trigger software actions at appropriate moments - fundamentally shifting from reactive systems that await explicit instructions. However, existing approaches lack generalizable end-to-end solutions for measuring and optimizing such anticipatory behaviors. This paper introduces ProActor, a unified framework for conversational task scheduling that integrates: (1) a domain-agnostic automated annotation methodology that enables scalable proactiveness reinforcement learning (RL) by generating full opportunity time windows instead of rigid point labels, (2) systematic proactiveness metrics capturing both timing quality and reference action alignment, and (3) RL optimization using GRPO with various reward designs. Our insight is that RULER-based rewards with proactiveness rubrics are crucial for improving timing quality, and that proactiveness optimization enabled by stage-aware composite rewards is key to balancing timing quality and reference action alignment. Timing-aware RL requires extensive exploration, demanding efficient infrastructure. We develop ART-F, an adaptive framework combining request-adaptive inference clusters with DDP-based training on single-node multi-GPU systems, enabling LoRA training of 4-bit Qwen2.5-14B-ProActor-Q4 with 4-8x speedups. Experiments on two newly auto-annotated datasets demonstrate significant improvements in proactive timing while maintaining action consistency comparable to state-of-the-art (SOTA) baselines. Ablations validate the effectiveness of distinct composite reward variations.

主动调度强化学习对话系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。