提前预警对话系统失败，让问题在崩溃前被发现。
TRACER: Early Failure Detection for Task-Oriented Dialogue
- 结合对话状态变化和文本表示，从部分对话中预测最终失败
- 在仅25%对话内容时已能有效检测失败信号
- 适合需要实时监控的对话系统部署场景
任务导向型对话系统常在最终崩溃前就已失败，但现有评估多在问题明显后才进行。我们提出TRACER，一种早期失败检测方法：通过融合信念状态变化轨迹与对话状态的文本表示，从部分对话中预测完整对话是否会失败。我们在理想信念状态和生成信念状态两种设置下评估，测试了仅可见25%、50%、75%或100%对话时的效果。结果表明，TRACER能在对话结束前较早识别出失败信号，并显著优于启发式、经典及单流基线模型。这些结果表明，早期失败检测可为对话系统提供实际的预警告信号。
原文摘要 · Abstract (English)
Task-oriented dialogue systems often fail before the final breakdown is obvious, but most evaluation only measures failure after the conversation has already gone wrong. We present TRACER, a method for early failure detection in task-oriented dialogue. TRACER predicts from a partial dialogue whether the full conversation will eventually fail by combining simple trajectory signals from belief-state changes with text representations of the evolving dialogue state. We evaluate the method in both oracle and generated belief-state settings, and test how well it works when only 25%, 50%, 75%, or 100% of the dialogue is visible. Across these settings, TRACER detects useful failure signals well before the end of the conversation and outperforms heuristic, classical, and single-stream baselines. These results suggest that early failure detection can provide a practical warning signal for dialogue systems before the interaction fully breaks down.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。