arXiv:2601.19120cs.IRcs.AI2026-01被引 1

首个评估大模型推荐解释鲁棒性的框架,揭示真实场景下解释稳定性问题。

RobustExplain: Evaluating Robustness of LLM-Based Explanation Agents for Recommendation

  • 设计五类真实用户行为扰动,模拟点击误操作等噪声数据
  • 多维度评估显示大模型解释稳定性仅中等,70B模型比7B高8%稳定
  • 为可信赖推荐系统提供关键鲁棒性基准,适合关注可信AI的开发者

大型语言模型(LLMs)在推荐系统中越来越多地用于生成自然语言解释,作为基于用户行为历史推理的解释代理。尽管已有研究关注固定输入下的解释流畅性和相关性,但对真实场景中用户行为噪声下解释鲁棒性的研究仍属空白。在实际网络平台中,由于误点击、时间不一致、缺失值和偏好演化,交互历史天然存在噪声,影响解释的稳定性和用户信任。本文提出RobustExplain,首个系统评估LLM生成推荐解释鲁棒性的框架。该框架引入五类真实用户行为扰动,并在多个严重程度下评估,采用多维度鲁棒性指标,涵盖语义、关键词、结构和长度一致性。目标是建立任务级评估范式与初始基线,而非覆盖所有LLM的全面排行榜。在四个代表性模型(7B–70B)上的实验表明,当前模型鲁棒性仅为中等水平,更大模型最高提升8%稳定性。结果建立了首个解释代理鲁棒性基准,强调鲁棒性是大规模可信代理型推荐系统的关键维度。

原文摘要 · Abstract (English)

Large Language Models (LLMs) are increasingly used to generate natural-language explanations in recommender systems, acting as explanation agents that reason over user behavior histories. While prior work has focused on explanation fluency and relevance under fixed inputs, the robustness of LLM-generated explanations to realistic user behavior noise remains largely unexplored. In real-world web platforms, interaction histories are inherently noisy due to accidental clicks, temporal inconsistencies, missing values, and evolving preferences, raising concerns about explanation stability and user trust. We present RobustExplain, the first systematic evaluation framework for measuring the robustness of LLM-generated recommendation explanations. RobustExplain introduces five realistic user behavior perturbations evaluated across multiple severity levels and a multi-dimensional robustness metric capturing semantic, keyword, structural, and length consistency. Our goal is to establish a principled, task-level evaluation framework and initial robustness baselines, rather than to provide a comprehensive leaderboard across all available LLMs. Experiments on four representative LLMs (7B--70B) show that current models exhibit only moderate robustness, with larger models achieving up to 8% higher stability. Our results establish the first robustness benchmarks for explanation agents and highlight robustness as a critical dimension for trustworthy, agent-driven recommender systems at web scale.

推荐系统大模型鲁棒性解释生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。