用注意力历史增强对话与推理，提升大模型多轮思考能力。
History-Aware Cross-Attention Reinforcement: Self-Supervised Multi Turn and Chain-of-Thought Fine-Tuning with vLLM
- 通过vLLM异步捕获每层每头的注意力权重，追踪对话历史。
- 自监督奖励函数融合全程对话与思维链中间步骤的注意力信号。
- 适合需要深度推理和长对话的场景，如智能助手、复杂问答。
我们提出CAGSR-vLLM-MTC，是自监督交叉注意力引导强化框架的扩展，基于高性能vLLM运行时，同时解决多轮对话与思维链推理问题。在原有单轮方法基础上，我们修改vLLM的C++/CUDA内核，异步捕获生成过程中的逐层、逐头交叉注意力权重。进一步将自监督奖励函数推广至整个对话历史及中间思维链步骤的注意力信号累积。讨论了实际权衡，包括基于熵的钳制机制以防止早期上下文注意力坍缩，并展望了多角色对话与层次化推理的未来方向。
原文摘要 · Abstract (English)
We present CAGSR-vLLM-MTC, an extension of our Self-Supervised Cross-Attention-Guided Reinforcement (CAGSR) framework, now implemented on the high-performance vLLM runtime, to address both multi-turn dialogue and chain-of-thought reasoning. Building upon our original single-turn approach, we first instrumented vLLM's C++/CUDA kernels to asynchronously capture per-layer, per-head cross-attention weights during generation. We then generalized our self-supervised reward function to accumulate attention signals over entire conversation histories and intermediate chain-of-thought steps. We discuss practical trade-offs, including an entropy-based clamping mechanism to prevent attention collapse on early context, and outline future directions for multi-party dialogues and hierarchical reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。