用隐空间匹配替代语言生成，让大模型推荐快10倍以上
Decoding in Latent Spaces for Efficient Inference in LLM-based Recommendation
- 直接在隐空间匹配用户偏好与候选项的内部表示，跳过逐词生成
- 比传统语言空间解码快10倍以上，且性能不降反升
- 适合追求高效推理的大模型推荐系统开发者
将大语言模型（LLM）用于生成式推荐虽表现优异，但因语言空间的自回归解码导致推理开销巨大。本文提出轻量隐空间解码（L2D），通过匹配测试序列隐藏状态（反映用户偏好）与训练序列隐藏状态（对应候选项），实现无需自回归过程的隐空间解码。该方法保持原有生成式微调范式，不改变模型结构，却显著降低计算成本。大量实验表明，L2D相较语言空间解码提速超10倍，同时性能持平或更优。
原文摘要 · Abstract (English)
Fine-tuning large language models (LLMs) for recommendation in a generative manner has delivered promising results, but encounters significant inference overhead due to autoregressive decoding in the language space. This work explores bypassing language-space decoding by directly matching candidate items with the LLM's internal thought representations in the latent space, eliminating the time-consuming autoregressive process to reduce computational costs. Towards this, we introduce Light Latent-space Decoding (L2D), an effective and efficient latent-space decoding method. L2D represents user-preferred items by using the hidden states of test sequences reflecting the LLM's internal thought, and obtains candidate item representations from the hidden states of training sequences labeled with the corresponding candidate items. It then matches the two types of representations to decode items, achieving latent-space decoding. In this way, it enables efficient decoding without altering the LLM's generative tuning paradigm, thereby preserving performance. Extensive empirical results demonstrate that L2D is more than 10x faster than language-space decoding while maintaining or enhancing performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。