arXiv:2508.20559cs.CLcs.IR2025-08被引 1

用生成模型实现高效实时查询摘要,提升搜索体验。

Leveraging Generative Models for Real-Time Query-Driven Text Summarization in Large-Scale Web Search

  • 轻量模型通过蒸馏与优化,实现高质量生成摘要。
  • 支持每秒处理5万次查询,平均延迟低于55毫秒。
  • 适合需要低延迟高精度摘要的工业级搜索系统。

在大规模网络搜索中,查询驱动文本摘要(QDTS)旨在根据用户查询从文档中生成简洁且信息丰富的摘要,对提升用户参与度和加速决策至关重要。传统抽取式摘要模型依赖候选片段排序,存在多阶段流程中的信息累积损失及架构瓶颈问题;且对复杂查询意图的语义理解不足。本文提出一种新框架,首次将生成模型应用于工业级实时QDTS。通过大模型蒸馏、监督微调、直接偏好优化与前瞻解码,将仅0.1B参数的轻量模型转化为领域专用摘要专家。在多个行业相关指标上优于现有生产基线,达到新最优水平。部署效率优异,仅需334块NVIDIA L20 GPU即可实现约5万次/秒的查询吞吐量,单次查询平均延迟在55毫秒以内。

原文摘要 · Abstract (English)

In the dynamic landscape of large-scale web search, Query-Driven Text Summarization (QDTS) aims to generate concise and informative summaries from textual documents based on a given query, which is essential for improving user engagement and facilitating rapid decision-making. Traditional extractive summarization models, based primarily on ranking candidate summary segments, have been the dominant approach in industrial applications. However, these approaches suffer from two key limitations: 1) The multi-stage pipeline often introduces cumulative information loss and architectural bottlenecks due to its weakest component; 2) Traditional models lack sufficient semantic understanding of both user queries and documents, particularly when dealing with complex search intents. In this study, we propose a novel framework to pioneer the application of generative models to address real-time QDTS in industrial web search. Our approach integrates large model distillation, supervised fine-tuning, direct preference optimization, and lookahead decoding to transform a lightweight model with only 0.1B parameters into a domain-specialized QDTS expert. Evaluated on multiple industry-relevant metrics, our model outperforms the production baseline and achieves a new state of the art. Furthermore, it demonstrates excellent deployment efficiency, requiring only 334 NVIDIA L20 GPUs to handle \textasciitilde50,000 queries per second under 55~ms average latency per query.

生成摘要实时推理轻量化模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。