用Transformer生成特定细胞类型的调控DNA序列，效率更高且更少记忆训练数据。
Continuous Diffusion Transformers for Designing Synthetic Regulatory Elements
- 用2D CNN编码器+Transformer去噪器替代U-Net，提升生成效率
- 13轮训练即达最优损失，记忆率从5.3%降至1.7%
- 结合奖励模型微调，预测活性提升38倍，效果真实可靠
我们提出一种参数高效的扩散Transformer（DiT），用于生成200bp的细胞类型特异性调控DNA序列。通过将DNA-Diffusion中的U-Net主干替换为配备2D CNN输入编码器的Transformer去噪器，模型在13个训练周期内达到U-Net的最佳验证损失（减少60倍），收敛误差降低39%，同时通过BLAT比对将生成序列对训练数据的记忆率从5.3%降至1.7%。消融实验表明，CNN编码器至关重要：无此结构时，无论使用何种位置编码，验证损失均上升70%。我们进一步采用基于Enformer的DDPO微调方法，使预测调控活性提升38倍。在独立预测任务中与DRAKES交叉验证的结果表明，性能提升反映的是真实的调控信号，而非奖励模型过拟合。
原文摘要 · Abstract (English)
We present a parameter-efficient Diffusion Transformer (DiT) for generating 200bp cell-type-specific regulatory DNA sequences. By replacing the U-Net backbone of DNA-Diffusion with a transformer denoiser equipped with a 2D CNN input encoder, our model matches the U-Net's best validation loss in 13 epochs (60$\times$ fewer) and converges 39% lower, while reducing memorization from 5.3% to 1.7% of generated sequences aligning to training data via BLAT. Ablations show the CNN encoder is essential: without it, validation loss increases 70% regardless of positional embedding choice. We further apply DDPO finetuning using Enformer as a reward model, achieving a 38$\times$ improvement in predicted regulatory activity. Cross-validation against DRAKES on an independent prediction task confirms that improvements reflect genuine regulatory signal rather than reward model overfitting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。