arXiv:2602.06470cs.CLcs.AI2026-02被引 1

用用户操作日志优化大模型,自动过滤噪声并提升响应质量。

Improve Large Language Model Systems with User Logs

  • 将用户日志转化为结构化规则与偏好对,统一处理反馈信号。
  • 在多个基准上超越RAG和记忆基线,显著提升响应效果与效率。
  • 适合关注模型持续学习与真实场景优化的研究者与工程师。

扩大训练数据和模型参数规模长期推动大语言模型(LLM)发展,但该范式正面临高质量数据稀缺与计算成本递增的瓶颈。因此,近期研究转向从真实部署中持续学习,利用用户交互日志获取真实的用户反馈与流程知识。然而,用户日志具有非结构化和噪声大的特点,传统LLM系统难以区分有效信号与噪声行为,且日志收集与模型优化间的偏差(如离策略问题)加剧了这一挑战。为此,我们提出UNO(User log-driveN Optimization),一个基于用户日志改进大模型系统(LLMsys)的统一框架。UNO首先将日志提炼为半结构化规则与偏好对,再通过查询-反馈驱动聚类管理数据异质性,并量化模型先验知识与日志数据之间的认知差距。该评估引导系统自适应过滤噪声反馈,并为日志中提取的主经验与反思经验构建不同模块,从而改善后续响应。大量实验表明,UNO在有效性与效率上均达到当前最优水平,显著优于检索增强生成(RAG)与记忆基线方法。代码已开源:https://github.com/bebr2/UNO。

原文摘要 · Abstract (English)

Scaling training data and model parameters has long driven progress in large language models (LLMs), but this paradigm is increasingly constrained by the scarcity of high-quality data and diminishing returns from rising computational costs. As a result, recent work is increasing the focus on continual learning from real-world deployment, where user interaction logs provide a rich source of authentic human feedback and procedural knowledge. However, learning from user logs is challenging due to their unstructured and noisy nature. Vanilla LLM systems often struggle to distinguish useful feedback signals from noisy user behavior, and the disparity between user log collection and model optimization (e.g., the off-policy optimization problem) further strengthens the problem. To this end, we propose UNO (User log-driveN Optimization), a unified framework for improving LLM systems (LLMsys) with user logs. UNO first distills logs into semi-structured rules and preference pairs, then employs query-and-feedback-driven clustering to manage data heterogeneity, and finally quantifies the cognitive gap between the model's prior knowledge and the log data. This assessment guides the LLMsys to adaptively filter out noisy feedback and construct different modules for primary and reflective experiences extracted from user logs, thereby improving future responses. Extensive experiments show that UNO achieves state-of-the-art effectiveness and efficiency, significantly outperforming Retrieval Augmented Generation (RAG) and memory-based baselines. We have open-sourced our code at https://github.com/bebr2/UNO .

大模型优化用户日志持续学习反馈机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。