Introduction to Stable Diffusion Latent Upscaler

Upscale generated images by 2x natively

Image by the author

The topic for today is about a latent diffusion-based upscaler model to upscale Stable Diffusion generated images by 2x natively. The model was trained by Katherine Crowson in collaboration with Stability AI.

As of version 0.13.0, the diffusers package officially supports the stabilityai/sd-x2-latent-upscaler model under the StableDiffusionLatentUpscalePipeline class.

One main advantage of this pipeline is that you can use the latent output from any StableDiffusionPipeline and pass it as input to the upscaler before decoding it with the desired VAE.

Alternatively, you can encode an existing image to latent space before passing it to the upscaler and decode the output with any VAE.

Let’s proceed to the next section to install all the necessary modules.

Setup

Before that, it is highly recommended to create a new virtual environment.

diffusers

Activate the virtual environment and install the standard diffusers package via the following command:

pip install diffusers

Alternatively, install the latest diffusers package as follows: