动态调整生成长度，让大模型更准确地生成带格式的文本。
Dynamic Infilling Anchors for Format-Constrained Generation in Diffusion Large Language Models
- 不依赖训练，通过动态估算结束锚点位置来调节生成长度。
- 在GSM8K和MATH上实现显著零样本提升，格式符合率与答案准确率双增。
- 适合需要严格结构输出的推理、代码生成等任务场景。
扩散语言模型（dLLMs）具备双向注意力和并行生成能力，能利用全局上下文，自然支持可解析的JSON或推理模板等格式约束任务。虽然固定锚点方法可强制格式，但常导致生成跨度僵硬，引发推理截断或冗余内容。为此，我们提出无需训练的动态填充锚点（DIA）方法，在迭代填充前动态估计结束锚点位置以调整生成长度。该灵活机制确保结构正确性和语义连贯性，避免了固定跨度方法的低效问题。在推理基准测试中，DIA显著提升了格式合规性与答案准确性，在GSM8K和MATH上实现显著零样本增益。结果表明，DIA是实现可靠、结构感知生成的一条稳健路径。
原文摘要 · Abstract (English)
Diffusion large language models (dLLMs) offer bidirectional attention and parallel generation, enabling them to exploit global context and naturally support format-constrained tasks like parseable JSON or reasoning templates. While straightforward fixed anchors can enforce such constraints, they often impose rigid spans, leading to truncated reasoning or redundant content. To overcome this, we propose Dynamic Infilling Anchors (DIA), a training-free method that dynamically estimates end-anchor positions to adjust generation length before iterative infilling. This flexible mechanism ensures structural correctness and semantic coherence, avoiding the inefficiencies of fixed-span methods. Experiments on reasoning benchmarks demonstrate that DIA substantially improves format compliance and answer accuracy, achieving significant zero-shot gains on GSM8K and MATH. These results establish DIA as a robust pathway toward reliable, structure-aware generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。