arXiv:2603.18074cs.LGcs.AI2026-03

轻量级微调让大模型更懂技术客服,提升决策稳定性和效率。

Lightweight Adaptation for LLM-based Technical Service Agent: Latent Logic Augmentation and Robust Noise Reduction

  • 通过隐式逻辑增强,挖掘人类示范背后的决策链条。
  • 构建多真实答案数据集,有效降低响应多样性带来的噪声。
  • 混合奖励机制减少算力消耗,适合实际部署的技术客服场景。

在复杂技术客服领域适配大语言模型时,受限于人类示范中缺乏显式的认知链以及有效响应的多样性带来的固有模糊性,导致智能体难以内化潜在决策逻辑并有效泛化。同时,标准训练范式带来的高资源与时间开销也阻碍了实际应用。为此,本文提出一个轻量级适配框架,包含三项关键贡献:(1)隐式逻辑增强:引入规划感知轨迹建模与决策推理增强,弥合表面监督与潜在决策逻辑之间的差距,强化监督微调对齐的稳定性;(2)鲁棒噪声抑制:通过双重过滤方法构建多真实答案数据集,验证多样化响应以降低噪声,捕捉语义多样性;(3)轻量级适配:设计混合奖励机制,融合基于LLM的判别器与轻量级相关性重排序器,提炼高保真奖励信号,相比标准的LLM-as-a-Judge强化学习显著降低计算成本。在真实云服务任务上的实证评估表明,该框架在语义多样场景下实现了决策稳定性与性能提升;同时,混合奖励机制在保持对齐效果的同时大幅缩短训练时间,凸显其在技术客服代理部署中的实用价值。

原文摘要 · Abstract (English)

Adapting Large Language Models in complex technical service domains is constrained by the absence of explicit cognitive chains in human demonstrations and the inherent ambiguity arising from the diversity of valid responses. These limitations severely hinder agents from internalizing latent decision dynamics and generalizing effectively. Moreover, practical adaptation is often impeded by the prohibitive resource and time costs associated with standard training paradigms. To overcome these challenges and guarantee computational efficiency, we propose a lightweight adaptation framework comprising three key contributions. (1) Latent Logic Augmentation: We introduce Planning-Aware Trajectory Modeling and Decision Reasoning Augmentation to bridge the gap between surface-level supervision and latent decision logic. These approaches strengthen the stability of Supervised Fine-Tuning alignment. (2) Robust Noise Reduction: We construct a Multiple Ground Truths dataset through a dual-filtering method to reduce the noise by validating diverse responses, thereby capturing the semantic diversity. (3) Lightweight Adaptation: We design a Hybrid Reward mechanism that fuses an LLM-based judge with a lightweight relevance-based Reranker to distill high-fidelity reward signals while reducing the computational cost compared to standard LLM-as-a-Judge reinforcement learning. Empirical evaluations on real-world Cloud service tasks, conducted across semantically diverse settings, demonstrate that our framework achieves stability and performance gains through Latent Logic Augmentation and Robust Noise Reduction. Concurrently, our Hybrid Reward mechanism achieves alignment comparable to standard LLM-as-a-judge methods with reduced training time, underscoring the practical value for deploying technical service agents.

大模型适配技术客服轻量级训练奖励机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。