Projects for the

MVA Course: Generative Modeling

Projects by Groups of 2 students.
Report:
- 4 to 8 pages in simple column format.
- Figures and References can be added in appendix (not counted in the 8 pages).
- Introduction should place the context and connect with the course sessions.
- The practical part should include experiments which are not in the original paper
  (other data, other inverse problem, relevant low-dimensional experiments, etc).
  Comparisons with methods seen in practice sessions is encouraged when relevant.
- The use of chatbots in your project work is not forbidden. However, if you use one
  you must devote some part of your report to describe your usage (examples of prompts,
  how you refined them, comments on possible good/false responses).
  In this case, in the evaluation, we will put the emphasis on the feedback that you
  bring w.r.t. the chatbot's responses.
Codes for your experiments should be sent as a zip archive (not including large database).
Each project comes with a suggestion of 3 work tracks (which you may follow, or not).
Defense: 15' presentation followed by 15' questions.
Project Defense at Télécom Paris between March 30th and April 1st.

Explain the differences between WGAN and MMD-GAN (for theory and training).
Explain the gradient bias problem that may happen with GANs or WGANs.
Train a MMD-GAN for a low-dimensional dataset (sklearn make_moons) and a large-scale dataset.

Compare the proposed generative algorithm with the DDPM scheme
Test the generative algorithm using the score network of the last practical session
Test the restoration algorithm with another denoiser (e.g. DRUNet)

In PnP split Gibbs sampling, can we use generative models with latent variables?
Compare PnP split Gibbs sampling with Chung et al., 2023 (TP6, Exo3).
Are there convergence guarantees for the PnP Gibbs sampling algorithm?

Experiments using the DDPM model used in TP6.
Compare theoretically and experimentally with Chung et al. ICLR 2023 (TP6, Exo 3) for inpainting and Gaussian deblurring.
Is the method stable in when the noise measurement increases for Gaussian deblurring?

Discuss the adequation between the algorithm and the theoretical results.
Ablation experiments: Can you assess what is the most important term for conditionning? Are they all necessary?
For super-resolution, compute the standard deviation of each pixel when running several times the algorithm and discuss the results.

Focus your discussions and experiments only on the case of discrete-time diffusion models and linear non blind inverse problems.

In the case of linear non blind inverse problems, explicit the algorithm steps and make a comparison with the DPS algorithm (Chung et al 2023) seen in class.
Test the proposed control in place of the one proposed by the DPS algorithm for deblurring and random inpainting on FFHQ? Is it well adapted to the stochastic DDPM sampler?
Test the code provided by the authors on large area inpainting (like squares in faces as done in DPS) on a few images. Are the results satisfying?

Train a diffusion model on a 2D toy dataset (eg sklearn make_moons), and then train a consistency model using the two different approaches (distillation of diffusion VS independent training).
For inpainting, discuss the quality of the blending between known and unknwon pixels.
Compare experimentally with Chung et al. 2023 (TP6).

Explain the method using Figure 1
Test on toy datasets (GMM (See appendix 8.4.1), in 2D for visualization of the steps 1,2,3.)
Compare with DPS

Explain the method
Compare to DPS
Test on toy datasets (GMM or two-moons/spiral, in 2D, for inpainting with $A = (1 0)$ for example.)

Explain the mathematical problem
Explain Theorem 4.1 and how the two given examples fit in
Test on toy datasets (GMM or two-moons/spiral, even on 2D datasets, with the examples below Theorem 4.1)

What is the performance of Algorithm 3 for image restoration if you use another deep neural network denoiser instead of the proposed flow denoiser?
Can you comment Proposition 4 and its proof? Especially, is the algorithm deterministic? Can you propose a reformulation of this proposition?
What are the experimental limits of PnP-Flow? Can you show experiments, where PnP-Flow fails to restore? (Suggestion : you can look at restoration with very noisy images or inpainting with large masked patches)

How does Flow Matching sampling compare to the diffusion models studied in class? What are the main differences and prove in what sense are they equivalent (or not).
Can you replicate Figure 1 on a toy dataset of your choice (e.g., random points, rings, mixtures)? How does the dimension dependence of the empirical “collapse time” compare to the theoretical collapse time predicted in Biroli et al.?
Can you replicate Figure 2.1 and Figure 3 on toy datasets of your choice, with the goal of identifying the generalization switching time?
Identify how the switching time varies with the dataset geometry and dimension, the flow matching time schedule, and other relevent parameters (model capacity, training time, etc.). Can you explain the trends observed experimentally?

In the Gaussian mixture case, does the equivalence between the optimal transport (OT) map and the flow matching map (proved for single Gaussians in Theorem 3) still hold? To investigate this, start with numerical experiments comparing OT and flow matching maps between a standard Gaussian and a discrete target measure (e.g., a sum of Dirac masses).
The paper states: “In these cases, it is evident that our goal is not to compute the velocity field exactly, but rather to rely on its approximation by a neural network, a key element which gives the model its generalization properties.” Comment on this claim both theoretically and numerically.
Construct an explicit counterexample with disconnected support where iterative rectification does not converge to the OT map, but smoothed rectification does.
On the same example, study and compare the convergence behavior of Diffusion Schrödinger Bridge Matching (Appendix F.2).