arXiv:2512.06690cs.CL2025-12被引 34

让AI边生成边思考,个性化长文本更自然高效

Think-While-Generating: On-the-Fly Reasoning for Personalized Long-Form Generation

  • 生成时同步进行推理,动态调整内容
  • 在真实数据集上个性化效果优于现有方法
  • 适合需要持续互动的对话与创作场景

偏好对齐使大模型更贴近人类期望,但现有方法多聚焦群体偏好,忽视个体差异。早期个性化方法如提示定制或微调难以捕捉隐式偏好,限制实际应用。近期“先思考再生成”方法虽有改进,但在长文本生成中面临静态一次性推理难以覆盖全篇内容的问题,导致学习困难且适应性差。为此,我们提出FlyThinker,一种高效的“边生成边思考”框架,通过独立的推理模型并行生成逐标记的隐含推理内容,并融合进生成模型以动态引导输出。该设计实现推理与生成并发,保障推理效率;同时推理模型仅依赖先前回复而非自身历史输出,确保训练时各位置可并行处理,所有推理标记可在单次前向传播中完成,保持训练效率。在真实世界基准上的大量实验表明,FlyThinker在保持高效训练与推理的同时,显著提升个性化长文本生成质量。代码已开源。

原文摘要 · Abstract (English)

Preference alignment has enabled large language models (LLMs) to better reflect human expectations, but current methods mostly optimize for population-level preferences, overlooking individual users. Personalization is essential, yet early approaches-such as prompt customization or fine-tuning-struggle to reason over implicit preferences, limiting real-world effectiveness. Recent "think-then-generate" methods address this by reasoning before response generation. However, they face challenges in long-form generation: their static one-shot reasoning must capture all relevant information for the full response generation, making learning difficult and limiting adaptability to evolving content. To address this issue, we propose FlyThinker, an efficient "think-while-generating" framework for personalized long-form generation. FlyThinker employs a separate reasoning model that generates latent token-level reasoning in parallel, which is fused into the generation model to dynamically guide response generation. This design enables reasoning and generation to run concurrently, ensuring inference efficiency. In addition, the reasoning model is designed to depend only on previous responses rather than its own prior outputs, which preserves training parallelism across different positions-allowing all reasoning tokens for training data to be produced in a single forward pass like standard LLM training, ensuring training efficiency. Extensive experiments on real-world benchmarks demonstrate that FlyThinker achieves better personalized generation while keeping training and inference efficiency. Our code is available at https://github.com/wcb0219-sketch/FlyThinker.git.

个性化生成长文本生成动态推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。