为心理脆弱用户阻断危险内容循环，推荐系统更安全。
First, do no harm: Breaking suicidogenic echo chambers in media recommendation
- 在现有模型上加层重排序，根据用户脆弱度惩罚危险内容、提升治疗性内容。
- 模拟显示危机期能有效阻止有害内容推荐，且准确率下降可控。
- 可调参数适配不同临床标准，适合心理健康相关推荐场景。
推荐系统通常以用户参与度为目标，但在心理健康领域存在风险。当用户表现出自杀倾向时，常规算法常将其困在有害内容的回音室中，加剧心理危机。为此，我们提出RankAid，一种重排序方法，在保持预测相关性的同时优先考虑临床安全。该方法作为附加层，根据用户当前脆弱程度对高风险项目施加惩罚，同时提升具有治疗价值的内容。我们在MovieLens 1M数据集上进行评估，使用大语言模型对项目进行临床风险与治疗价值的语义标注。模拟结果显示，该算法能在危机高峰期成功阻止有害内容推荐，并主动重塑推荐流以促进情绪缓解。此外，这一安全干预仅导致标准准确率指标（如NDCG）出现可控、可接受的下降。通过使用非对称超参数，系统管理员还可根据具体临床指南灵活调节干预强度。
原文摘要 · Abstract (English)
Recommender systems generally optimises user engagement, but this approach is dangerous in mental health contexts. When vulnerable users show signs of suicidal ideation, standard algorithms often trap them in echo chambers of harmful content, worsening their psychological state. In response, we introduce RankAid, a re-ranking method that prioritises clinical safety alongside predictive relevance. It works as an add-on layer to existing models: it penalises risky items and boosts therapeutic content depending on the user's current level of vulnerability. We evaluated this approach using the MovieLens 1M dataset, where items were semantically annotated for clinical risk and therapeutic value using large language models. Our simulations show that our algorithm successfully blocks the recommendation of harmful content during crisis peaks, actively reshaping the feed to support emotional de-escalation. Furthermore, this safety intervention only causes a controlled, acceptable drop in standard accuracy metrics like NDCG. By using asymmetric hyperparameters, RankAid also gives system administrators the flexibility to tune the severity of the intervention based on specific clinical guidelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。