arXiv:2505.09925cs.LGcs.AI2025-05被引 4

让AI实时学习人类反馈,边学边抗干扰。

Reinforced Interactive Continual Learning via Real-time Noisy Human Feedback

  • 用LLM动态接收实时人类反馈,持续更新模型。
  • 在含噪数据上仍保持高准确率,优于现有方法。
  • 适合需要人机交互的智能系统研发者。

本文提出一种交互式持续学习新范式,使AI模型能从实时人类反馈中动态学习新技能,同时保留已有知识。该范式突破传统持续学习的两大局限:一是采用流式、实时标注的数据,而非静态固定标签数据集;二是显式处理真实交互中常见的噪声反馈。为此,我们设计RiCL框架,利用大语言模型从动态反馈中高效学习。其包含三个核心组件:基于时间一致性的净化器,自动区分数据流中的干净与噪声样本;交互感知的直接偏好优化策略,协调模型生成与人类反馈的一致性;抗噪声对比学习模块,通过挖掘数据内在关系构建鲁棒表征,避免依赖不可靠标签。在两个基准数据集FewRel和TACRED上,引入真实噪声模式的实验表明,RiCL显著优于当前最先进的在线持续学习与噪声标签学习方法组合。

原文摘要 · Abstract (English)

This paper introduces an interactive continual learning paradigm where AI models dynamically learn new skills from real-time human feedback while retaining prior knowledge. This paradigm distinctively addresses two major limitations of traditional continual learning: (1) dynamic model updates using streaming, real-time human-annotated data, rather than static datasets with fixed labels, and (2) the assumption of clean labels, by explicitly handling the noisy feedback common in real-world interactions. To tackle these problems, we propose RiCL, a Reinforced interactive Continual Learning framework leveraging Large Language Models (LLMs) to learn new skills effectively from dynamic feedback. RiCL incorporates three key components: a temporal consistency-aware purifier to automatically discern clean from noisy samples in data streams; an interaction-aware direct preference optimization strategy to align model behavior with human intent by reconciling AI-generated and human-provided feedback; and a noise-resistant contrastive learning module that captures robust representations by exploiting inherent data relationships, thus avoiding reliance on potentially unreliable labels. Extensive experiments on two benchmark datasets (FewRel and TACRED), contaminated with realistic noise patterns, demonstrate that our RiCL approach substantially outperforms existing combinations of state-of-the-art online continual learning and noisy-label learning methods.

持续学习人机交互噪声鲁棒

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。