Official PyTorch implementation of our paper An Inversion-based Measure of Memorization for Diffusion Models. Note that we are actively improving our paper (till it get published), so that there might be some inconsistence between this updated repository and the results presented in the old version of paper.
This repository follows the implemetations of codebases pytorch-ddpm, latent-diffusion, stable-diffusion-v1, stable-diffusion-v2 and stable-diffusion-v3. We also modify some source codes, including introducing xformers support for stable-diffusion-v1 and fixing bugs in DDIM sampler.
Go to the specific directory and create an anaconda environment with:
cd ddpm[latent-diffusion, stable-diffusion-v1, stable-diffusion-v2, stable-diffusion-v3]
conda env create -f environment.yamlChange Torch and xformers to appropriate versions depending on your own CUDA run time library.
CIFAR-10 can be downloaded on the official website. Obtain the IDs of 99 highly memorized images and 1000 normal images if needed.
We use the CelebAHQ-256 dataset on Kaggle and FFHQ following their official repository.
The subsets of LAION used in the paper can be downloaded here.
| Model | Dataset | Link |
|---|---|---|
| DDPM | CIFAR-10 | https://drive.google.com/file/d/1TJDmFdb6-ZwI2AqOfTCClNWn_iAGjdvN/view?usp=sharing |
| LDM | CelebAHQ FFHQ | https://drive.google.com/drive/folders/1eeO9E4zLTdy1PfPA55YhIwclS9XBF-UI?usp=sharing |
| SD v1.4 | LAION Subset | https://drive.google.com/drive/folders/1TNvSc6JMvCqZJ4-9FO-A4-bwjYVReOIc?usp=sharing |
| SD v2.1 | LAION Subset | https://drive.google.com/drive/folders/1qiFMpUfLdZdLWRV-TkmPJ1-AEMmUsF07?usp=sharing |
| SD v3.5 | LAION Subset | https://drive.google.com/drive/folders/1bjXDH8iQOb5F-ADBBModB_vmIMrGbhvg?usp=sharing |
We use SSCD to calculate image similarity. Download the sscd_disc_large model in their official repository.
Download our pretrained DDPM and run the following command to perform inversion and calculate memorization scores:
python inversion.py --logdir logs/DDPM_CIFAR10_EPS_INVERSIONWe provide pretrained models on the subsets of CelebAHQ and FFHQ. Pretrained models on the full datasets can be found in the official repository.
To perform inversion:
python inversion -dp /path/to/dataset --ckpt_file /path/to/pretrained_modelDownload pretrained SD v1.4, SD v2.1 and SD v3.5, and then run:
python inversion -dp /path/to/datasetEach model follows their original license.