DrDiff通过动态调度与分层注意力，打破长文本生成的效率与质量矛盾。
DrDiff: Dynamic Routing Diffusion with Hierarchical Attention for Breaking the Efficiency-Quality Trade-off
- 动态专家调度按文本复杂度分配算力，智能优化资源使用。
- 分层稀疏注意力将计算复杂度从O(n²)降至O(n)，保持性能不降。
- 结合DPM-solver++加速扩散过程，显著提升生成速度，适合长文本场景。
本文提出DrDiff，一种新型长文本生成框架，通过三项核心技术突破效率与质量之间的权衡。首先，设计动态专家调度机制，在扩散过程中根据文本复杂度智能分配计算资源，实现对不同难度任务的高效处理。其次，引入分层稀疏注意力（HSA）机制，自适应调整注意力模式以应对多种输入长度，将计算复杂度从O(n²)降低至O(n)，同时维持模型性能。最后，提出软吸收引导优化策略，与DPM-solver++结合，减少扩散步数，显著提升生成速度。在多个长文本生成基准上的全面实验表明，DrDiff优于现有最先进方法。
原文摘要 · Abstract (English)
This paper introduces DrDiff, a novel framework for long-text generation that overcomes the efficiency-quality trade-off through three core technologies. First, we design a dynamic expert scheduling mechanism that intelligently allocates computational resources during the diffusion process based on text complexity, enabling more efficient handling of text generation tasks of varying difficulty. Second, we introduce a Hierarchical Sparse Attention (HSA) mechanism that adaptively adjusts attention patterns according to a variety of input lengths, reducing computational complexity from O($n^2$) to O($n$) while maintaining model performance. Finally, we propose a soft absorption guidance optimization strategy that combines with DPM-solver++ to reduce diffusion steps, significantly improving generation speed. Comprehensive experiments on various long-text generation benchmarks demonstrate the superiority of our DrDiff over the existing SOTA methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。