用大模型把音乐推荐变成生成任务,对话更自然、推荐更准。
TALKPLAY: Multimodal Music Recommendation with Large Language Models
- 把推荐问题转为大模型的文本生成任务,统一处理对话与推荐。
- 在多轮对话中保持高推荐准确率,优于仅依赖文本或听歌历史的方法。
- 适合需要自然语言交互的智能音乐推荐场景。
我们提出 TALKPLAY,一种基于大语言模型(LLM)的多模态音乐推荐系统,将推荐任务重构为令牌生成问题。通过利用 LLM 的指令遵循与自然语言生成能力,该系统能根据多样化的用户查询推荐音乐,并生成上下文相关的回应。尽管预训练的 LLM 主要针对文本模态,TALKPLAY 通过两项关键创新扩展其应用:一是多模态音乐分词器,可编码音频特征、歌词、元数据、语义标签及播放列表共现信号;二是词汇扩展机制,实现语言与音乐相关令牌的统一处理与生成。通过将推荐系统直接嵌入 LLM 架构,TALKPLAY 实现了三方面改进:(1) 将传统的两阶段对话推荐系统(推荐引擎与对话管理器)整合为端到端统一系统;(2) 有效利用长对话上下文进行推荐,同时在多轮交互中保持强性能;(3) 生成自然语言响应,实现无缝用户交互。定性与定量评估表明,TALKPLAY 在推荐性能和对话自然度上显著优于仅依赖文本或听歌历史的单模态方法。
原文摘要 · Abstract (English)
We present TALKPLAY, a novel multimodal music recommendation system that reformulates recommendation as a token generation problem using large language models (LLMs). By leveraging the instruction-following and natural language generation capabilities of LLMs, our system effectively recommends music from diverse user queries while generating contextually relevant responses. While pretrained LLMs are primarily designed for text modality, TALKPLAY extends their scope through two key innovations: a multimodal music tokenizer that encodes audio features, lyrics, metadata, semantic tags, and playlist co-occurrence signals; and a vocabulary expansion mechanism that enables unified processing and generation of both linguistic and music-relevant tokens. By integrating the recommendation system directly into the LLM architecture, TALKPLAY transforms conventional systems by: (1) unifying previous two-stage conversational recommendation systems (recommendation engines and dialogue managers) into a cohesive end-to-end system, (2) effectively utilizing long conversational context for recommendation while maintaining strong performance in extended multi-turn interactions, and (3) generating natural language responses for seamless user interaction. Our qualitative and quantitative evaluation demonstrates that TALKPLAY significantly outperforms unimodal approaches based solely on text or listening history in both recommendation performance and conversational naturalness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。