The Power of VAEs in Stable Diffusion: Install Guide

Updated
The Power of VAEs in Stable Diffusion: Install Guide
Variable Auto Encoder, abbreviated as VAE, is a term used to describe files that complement your Stable Diffusion checkpoint models, enhancing the vividness of colors and the sharpness of images. VAEs bring an additional advantage of improving the depiction of hands and faces. In case you encounter washed-out images, it is advisable to download a VAE to address this issue effectively.

1. Introduction

Welcome to another exciting tutorial on Stable Diffusion! In this comprehensive guide, we will delve into the process of downloading and installing VAEs (Variational Autoencoders) to enrich the overall rendering quality, not limited to just eyes and faces. By harnessing the power of VAEs, you will unlock a world of captivating visual effects, enhancing the realism and intricate details in all your Stable Diffusion projects. Get ready to take your creations to new heights!

2. What is a VAE in Stable Diffusion?

A VAE, which stands for Variational Autoencoder, is a crucial component of the neural network model used in Stable Diffusion. It encodes and decodes images to and from a smaller latent space, allowing for faster computation.

What is the effect of using VAE?

The effect of using VAE is generally subtle but impactful. An improved VAE enhances the decoding process and leads to better image recovery, particularly for fine details such as eyes and text.

Should I use a VAE?

If you come across washed-out images, it is recommended to download a VAE to effectively tackle this problem and restore the image's vibrancy. Below a couple examples where you could clearly see the difference between image generation with and without the use of an VAE.

Example 1 (NOT using a VAE): the-power-of-vaes-in-stable-diffusion-install-guide.webp

Example 2 (USING a VAE): the-power-of-vaes-in-stable-diffusion-install-guide_2.webp

  • Positive Prompt: ((detailed)), ((masterpiece)), ((best quality)), ((Astounding)), ((Enchanting)), (Striking), (lara croft:1.2), (tomb raider:1.2), (tomb:1.2), cave, woman, (inside tomb:1.1), (upper body:1.4), waist and up, (action pose:1.3), (detailed face:1.2), (looking at camera:1.3), skin pores, detailed eyes, clear skin, detailed pupils, (looking in to camera:1.1), close up
  • Negative Prompt: (worst quality:1.2), (low quality:1.2), (lowres:1.1), (monochrome:1.1), (greyscale), multiple views, comic, sketch, (((bad anatomy))), (((deformed))), (((disfigured))), watermark,(blurry), (((strabismus))), (wrong finger), multiple_views, mutation hands, mutation fingers, extra fingers, missing fingers, watermark, NG_DeepNegative_V1_75T, nude, nsfw, , nsfw, nude, bad hans, no hads, misplaced fingers
  • Sampling Steps: 30
  • Sampler: DDIM
  • CFG scale: 7
  • Size: 704x392
  • Model: fantasticmix
  • Seed: 2879321373

In the above examples you can clearly see a difference between the generated images. The first example looks really washed-out compared to example 2, where a VAE is used. Using a VAE is of course a personal choice based on your preferences and the results you desire.

If you strive for continuous improvement and seek the best possible outcomes, incorporating a VAE into your workflow can be beneficial. However, if you are satisfied with your current results, it is not necessary to use a VAE. You can just use the downloaded checkpoint/model as your base. Below is a comprehensive guide available for you to explore.

How to Install Stable Diffusion Checkpoints & Models

Stable Diffusion in the Cloud⚡️Run Automatic1111 in your browser in under 90 seconds
20% bonus on first deposit

3. How to Use VAEs in Stable Diffusion

At present, Stability has released two enhanced versions of VAE. Here are the direct links for downloading them.

Direct Download VAE Links (.safetensors files):

File Placement: For the AUTOMATIC1111 GUI, place the downloaded VAE files in the following directory: stable-diffusion-webui/models/VAE: stablediffusion_vae_model_folder_structure_nextdiffusion.png

How to use the downloaded VAE model?

  • Navigate to the Settings tab
  • Click on the Stable Diffusion section
  • Look for the section called SD VAE
  • Select the desired/downloaded VAE file from the dropdown menu
  • Finally click the Apply Settings button to load the VAE VAE_setttings_stable_diffusion_sd_vae_nextdiffusion.png

4. TIP (Adding VAEs to top of page)

TIP: Would you like to have your VAE models appear at the top of page, next to your checkpoints/models?

  • Navigate to the Settings tab
  • Click on the User Interface section
  • Look for the section called Quicksettings list
  • Search for "sd_vae" and add it

sd_vae_to_quicksettings_list_stablediffusion_nextdiffusion.png

When you have added the "sd_vae" to the Quicksettings you can click Apply Settings and Reload UI at the top of the page. After the UI is reloaded you should see the VAE dropdown menu at the top of the page: vae_added_to_top_of_stablediffusion_ui_nextdiffusion.png

Stable Diffusion in the Cloud⚡️Run Automatic1111 in your browser in under 90 seconds

5. Conclusion

In conclusion, VAEs enhance the visual quality of Stable Diffusion checkpoint models by improving image sharpness, color vividness, and the depiction of hands and faces. Downloading a VAE is an effective solution for addressing washed-out images. Incorporating VAEs into your workflow can lead to continuous improvement and better results. Explore the possibilities and elevate your Stable Diffusion experience with Variational Auto Encoders (VAEs).