arXiv:2606.00011cs.HCcs.AI2026-06被引 1

让医生能提前预判模型修改的失败风险并看到效果,提升人机协作精准度。

RuleEdit: Failure-Guided Human-AI Model Editing with Prospective Impact Preview

论文配图:RuleEdit: Failure-Guided Human-AI Model Editing with Prospective Impact Preview
图 1 · 摘自论文原文
  • 用规则表生成可解释的错误信号,识别模型潜在失效点。
  • 用户反馈后性能提升36.38%,错误拒绝率显著下降。
  • 适合医疗等高风险领域,需谨慎权衡局部优化与全局影响。

尽管人工智能在复杂决策中展现潜力,但从业者仍缺乏检测潜在失败及预览模型修改后果的工具。我们提出RuleEdit,一种交互式、规则引导的人机模型编辑系统:(i) 通过规则表生成可解释的不匹配信号,揭示可能的失败;(ii) 支持用户自定义规则反馈,并提供性能变化与嵌入空间迁移的前瞻性预览。我们在卒中康复评估场景中,由医护人员和学生进行评估。规则引导的故障检测使人机协同性能提升14.16%(p<0.001),同时减少对错误AI的依赖,降低过度和不足依赖,以及错误修正决策。此外,展示前瞻性嵌入预览后,用户反馈质量提高,模型更新后的局部性能提升从11.50%增至36.38%(p<0.001)。研究发现,基于不匹配的故障提示与前瞻性影响预览有助于实现故障感知的人机协同编辑,但也揭示了局部-全局权衡:针对特定案例的优化可能损害整体表现。本文讨论了设计故障感知与可控人机系统的设计启示。

原文摘要 · Abstract (English)

Despite the promise of AI to assist complex decisions, practitioners still lack ways to detect likely failures and inspect the consequences of model edits before committing them. We present RuleEdit, an interactive, rule-guided human-AI model editing system that (i) surfaces likely failures through interpretable mismatch signals from rule tables and (ii) supports user-authored rule feedback with prospective previews of projected performance changes and embedding shifts. We instantiate RuleEdit in stroke rehabilitation assessment and evaluate it with health professionals and students. Rule-guided failure detection significantly increased Human + AI performance by 14.16\% ($p<0.001$) while improving rejection of incorrect AI and reducing both over- and under- reliance as well as ChangedToWrong decisions. In addition, presenting prospective embedding previews improved participants' feedback for model adaptation, increasing post-update local performance gains from 11.50\% to 36.38\% after incorporating users' rule-based feedback ($p<0.001$). Our findings show that mismatch-based failure cues and prospective impact previews can support failure-aware human-AI model editing, while also revealing a local-global tradeoff: edits that help a specific case can degrade performance when transferred globally. We discuss implications of designing failure-aware and controllable human-AI systems.

人机协作模型编辑医疗AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。