arXiv:2603.05134cs.CLcs.AI2026-03被引 3

用分层大模型提升广告竞价智能性,兼顾推理与精准执行

LBM: Hierarchical Large Auto-Bidding Model via Reasoning and Acting

  • 分两层设计:高层推理+低层执行,结合语言与数值输入
  • 离线强化微调避免幻觉,训练效率高且泛化能力强
  • 适合需要智能竞价策略的广告平台与算法工程师

在线广告平台广告拍卖规模不断增大,人工竞价已不现实,需依赖自动竞价实现经济目标。现有自动竞价方法多采用离线强化学习或生成式方法优化策略,但因黑箱训练和数据集模式覆盖有限,常出现反直觉行为,难以理解任务状态且在动态环境中泛化能力差。大型语言模型(LLMs)凭借人类先验知识和推理能力,为提升自动竞价性能提供了新路径。然而直接应用LLM于竞价面临挑战:拍卖中需精确动作,而通用模型缺乏专业竞价知识,易产生幻觉并导致次优决策。为此,我们提出分层大型自动竞价模型(LBM),利用LLM的推理能力构建更优竞价策略。包括高层的LBM-Think模型用于推理,低层的LBM-Act模型用于动作生成。具体地,提出双嵌入机制,高效融合语言与数值输入,支持语言引导的训练;并引入一种名为GQPO的离线强化微调技术,缓解LBM-Think的幻觉问题,提升决策性能,无需模拟或真实投放。实验表明,基于LBM的生成式主干模型在高效训练和泛化能力方面表现优异。

原文摘要 · Abstract (English)

The growing scale of ad auctions on online advertising platforms has intensified competition, making manual bidding impractical and necessitating auto-bidding to help advertisers achieve their economic goals. Current auto-bidding methods have evolved to use offline reinforcement learning or generative methods to optimize bidding strategies, but they can sometimes behave counterintuitively due to the black-box training manner and limited mode coverage of datasets, leading to challenges in understanding task status and generalization in dynamic ad environments. Large language models (LLMs) offer a promising solution by leveraging prior human knowledge and reasoning abilities to improve auto-bidding performance. However, directly applying LLMs to auto-bidding faces difficulties due to the need for precise actions in competitive auctions and the lack of specialized auto-bidding knowledge, which can lead to hallucinations and suboptimal decisions. To address these challenges, we propose a hierarchical Large autoBidding Model (LBM) to leverage the reasoning capabilities of LLMs for developing a superior auto-bidding strategy. This includes a high-level LBM-Think model for reasoning and a low-level LBM-Act model for action generation. Specifically, we propose a dual embedding mechanism to efficiently fuse two modalities, including language and numerical inputs, for language-guided training of the LBM-Act; then, we propose an offline reinforcement fine-tuning technique termed GQPO for mitigating the LLM-Think's hallucinations and enhancing decision-making performance without simulation or real-world rollout like previous multi-turn LLM-based methods. Experiments demonstrate the superiority of a generative backbone based on our LBM, especially in an efficient training manner and generalization ability.

自动竞价大模型分层建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。