扩散模型-2020-理论基础：DDPM【目前“文本生图像”所采用的扩散模型大都是来自于DDPM】【输入：带噪音的图片+文本+噪音程度值；输出：待去除的噪音】【带噪音的图片-输出的噪音=生成的图片】

￡神魔★判官ぃ 2023-09-27 18:05 20阅读 0赞

原始论文：[Denoising Diffusion Probabilistic Models][]

分析论文：[Understanding Diffusion Models: A Unified Perspective][Understanding Diffusion Models_ A Unified Perspective]

分析论文：[The Curious Case of Neural Text Degeneration][]

分析论文：[Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions][]

分析论文：[WaveGrad: Estimating Gradients for Waveform Generation][WaveGrad_ Estimating Gradients for Waveform Generation]

分析论文：[Diffusion-LM Improves Controllable Text Generation][]

分析论文：[DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models][DiffuSeq_ Sequence to Sequence Text Generation with Diffusion Models]

分析论文：[Structured Denoising Diffusion Models in Discrete State-Spaces][]

分析论文：[Mask-Predict: Parallel Decoding of Conditional Masked Language Models - ACL Anthology][Mask-Predict_ Parallel Decoding of Conditional Masked Language Models - ACL Anthology]

分析论文：[MaskGIT: Masked Generative Image Transformer][MaskGIT_ Masked Generative Image Transformer]

分析论文：[Muse: Text-To-Image Generation via Masked Generative Transformers][Muse_ Text-To-Image Generation via Masked Generative Transformers]

近段时间最火的方向无疑是**基于文本用AI生成图像**，继OpenAI在2021提出的文本转图像模型[DALLE][]之后，越来越多的大公司卷入这个方向，如谷歌在今年相继推出了[Imagen][]和[Parti][]。一些主流的文本转图像模型如[DALL·E 2][DALL_E 2]，[stable-diffusion][]和[Imagen][]采用了**扩散模型**（**Diffusion Model**）作为图像生成模型，这也引发了对扩散模型的研究热潮。相比GAN来说，扩散模型训练更稳定，而且能够生成更多样的样本，OpenAI的论文[Diffusion Models Beat GANs on Image Synthesis][]也证明了扩散模型能够超越GAN。简单来说，扩散模型包含两个过程：**前向扩散过程**和**反向生成过程**，前向扩散过程是对一张图像逐渐添加高斯噪音直至变成**随机噪音**，而反向生成过程是**去噪音过程**，我们将从一个随机噪音开始逐渐去噪音直至生成一张图像，这也是我们要求解或者训练的部分。扩散模型与其它主流生成模型的对比如下所示：

![format_png][]

![a34cd38561204e2b85dfd816be5f8a36.png][]

![4f6f07f8f0ed4c0fad39399e6c7c239d.png][]

目前所采用的扩散模型大都是来自于2020年的工作[DDPM: Denoising Diffusion Probabilistic Models][DDPM_ Denoising Diffusion Probabilistic Models]**，**DDPM对之前的扩散模型（具体见[Deep Unsupervised Learning using Nonequilibrium Thermodynamics][]）进行了简化，并通过**变分推断**（variational inference）来进行建模，这主要是因为扩散模型也是一个**隐变量模型**（latent variable model），相比VAE这样的隐变量模型，扩散模型的隐变量是和原始数据是同维度的，而且推理过程（即扩散过程）往往是固定的。这篇文章将基于DDPM详细介绍扩散模型的原理，并给出具体的代码实现和分析。

### **扩散模型原理** ###

扩散模型包括两个过程：**前向过程（forward process）**和**反向过程（reverse process）**，其中前向过程又称为**扩散过程（diffusion process）**，如下图所示。无论是前向过程还是反向过程都是一个**参数化的马尔可夫链（Markov chain）**，其中反向过程可以用来生成数据，这里我们将通过变分推断来进行建模和求解。

![format_png 1][]

####  ####

--------------------

[扩散模型之DDPM - 知乎][DDPM -]

[https://www.youtube.com/watch?v=azBugJzmz-o][https_www.youtube.com_watch_v_azBugJzmz-o]

[【生成模型】DDPM概率扩散模型（原理+代码)\_杀生丸学AI的博客-CSDN博客][DDPM_AI_-CSDN]

[Denoising Diffusion Probabilistic Models]: https://arxiv.org/abs/2006.11239
[Understanding Diffusion Models_ A Unified Perspective]: https://arxiv.org/abs/2208.11970
[The Curious Case of Neural Text Degeneration]: https://arxiv.org/abs/1904.09751
[Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions]: https://arxiv.org/abs/1712.05884
[WaveGrad_ Estimating Gradients for Waveform Generation]: https://arxiv.org/abs/2009.00713
[Diffusion-LM Improves Controllable Text Generation]: https://arxiv.org/abs/2205.14217
[DiffuSeq_ Sequence to Sequence Text Generation with Diffusion Models]: https://arxiv.org/abs/2210.08933
[Structured Denoising Diffusion Models in Discrete State-Spaces]: https://arxiv.org/abs/2107.03006
[Mask-Predict_ Parallel Decoding of Conditional Masked Language Models - ACL Anthology]: https://aclanthology.org/D19-1633/
[MaskGIT_ Masked Generative Image Transformer]: https://arxiv.org/abs/2202.04200
[Muse_ Text-To-Image Generation via Masked Generative Transformers]: https://arxiv.org/abs/2301.00704
[DALLE]: https://link.zhihu.com/?target=https%3A//openai.com/blog/dall-e/
[Imagen]: https://link.zhihu.com/?target=https%3A//arxiv.org/abs/2205.11487
[Parti]: https://link.zhihu.com/?target=https%3A//arxiv.org/abs/2206.10789
[DALL_E 2]: https://link.zhihu.com/?target=https%3A//openai.com/dall-e-2/
[stable-diffusion]: https://link.zhihu.com/?target=https%3A//github.com/CompVis/stable-diffusion
[Diffusion Models Beat GANs on Image Synthesis]: https://link.zhihu.com/?target=https%3A//arxiv.org/abs/2105.05233
[format_png]: https://img-blog.csdnimg.cn/img_convert/2e0464cbc75ebfbabcbed30cbdf49ffb.webp?x-oss-process=image/format,png
[a34cd38561204e2b85dfd816be5f8a36.png]: https://img-blog.csdnimg.cn/a34cd38561204e2b85dfd816be5f8a36.png
[4f6f07f8f0ed4c0fad39399e6c7c239d.png]: https://img-blog.csdnimg.cn/4f6f07f8f0ed4c0fad39399e6c7c239d.png
[DDPM_ Denoising Diffusion Probabilistic Models]: https://link.zhihu.com/?target=https%3A//arxiv.org/abs/2006.11239
[Deep Unsupervised Learning using Nonequilibrium Thermodynamics]: https://link.zhihu.com/?target=https%3A//arxiv.org/abs/1503.03585
[format_png 1]: https://img-blog.csdnimg.cn/img_convert/9c0d4d4fe6e77bf6badac51127c214b8.webp?x-oss-process=image/format,png
[DDPM -]: https://zhuanlan.zhihu.com/p/563661713
[https_www.youtube.com_watch_v_azBugJzmz-o]: https://www.youtube.com/watch?v=azBugJzmz-o
[DDPM_AI_-CSDN]: https://blog.csdn.net/qq_45752541/article/details/127956235