The Power of VAEs in Stable Diffusion: Install Guide

Welcome to another exciting tutorial on Stable Diffusion! In this comprehensive guide, we will delve into the process of downloading and installing VAEs (Variational Autoencoders) to enrich the overall rendering quality, not limited to just eyes and faces. By harnessing the power of VAEs, you will unlock a world of captivating visual effects, enhancing the realism and intricate details in all your Stable Diffusion projects. Get ready to take your creations to new heights!

A VAE, which stands for Variational Autoencoder, is a crucial component of the neural network model used in Stable Diffusion. It encodes and decodes images to and from a smaller latent space, allowing for faster computation.

What is the effect of using VAE?

The effect of using VAE is generally subtle but impactful. An improved VAE enhances the decoding process and leads to better image recovery, particularly for fine details such as eyes and text.

Should I use a VAE?

If you come across washed-out images, it is recommended to download a VAE to effectively tackle this problem and restore the image's vibrancy. Below a couple examples where you could clearly see the difference between image generation with and without the use of an VAE.

Example 1 (NOT using a VAE):

Example 2 (USING a VAE):

• Positive Prompt: ((detailed)), ((masterpiece)), ((best quality)), ((Astounding)), ((Enchanting)), (Striking), (lara croft:1.2), (tomb raider:1.2), (tomb:1.2), cave, woman, (inside tomb:1.1), (upper body:1.4), waist and up, (action pose:1.3), (detailed face:1.2), (looking at camera:1.3), skin pores, detailed eyes, clear skin, detailed pupils, (looking in to camera:1.1), close up
• Negative Prompt: (worst quality:1.2), (low quality:1.2), (lowres:1.1), (monochrome:1.1), (greyscale), multiple views, comic, sketch, (((bad anatomy))), (((deformed))), (((disfigured))), watermark,(blurry), (((strabismus))), (wrong finger), multiple_views, mutation hands, mutation fingers, extra fingers, missing fingers, watermark, NG_DeepNegative_V1_75T, nude, nsfw, , nsfw, nude, bad hans, no hads, misplaced fingers
• Sampling Steps: 30
• Sampler: DDIM
• CFG scale: 7
• Size: 704x392
• Model: fantasticmix
• Seed: 2879321373

In the above examples you can clearly see a difference between the generated images. The first example looks really washed-out compared to example 2, where a VAE is used. Using a VAE is of course a personal choice based on your preferences and the results you desire.

If you strive for continuous improvement and seek the best possible outcomes, incorporating a VAE into your workflow can be beneficial. However, if you are satisfied with your current results, it is not necessary to use a VAE. You can just use the downloaded checkpoint/model as your base. Below is a comprehensive guide available for you to explore.

How to Install Stable Diffusion Checkpoints & Models

At present, Stability has released two enhanced versions of VAE. Here are the direct links for downloading them.

Direct Download VAE Links (.safetensors files):

• Download link for EMA VAE
• Download link for MSE VAE

File Placement: For the AUTOMATIC1111 GUI, place the downloaded VAE files in the following directory: stable-diffusion-webui/models/VAE:

How to use the downloaded VAE model?

• Navigate to the Settings tab
• Click on the Stable Diffusion section
• Look for the section called SD VAE
• Select the desired/downloaded VAE file from the dropdown menu
• Finally click the Apply Settings button to load the VAE

TIP: Would you like to have your VAE models appear at the top of page, next to your checkpoints/models?

• Navigate to the Settings tab
• Click on the User Interface section
• Look for the section called Quicksettings list
• Search for "sd_vae" and add it

When you have added the "sd_vae" to the Quicksettings you can click Apply Settings and Reload UI at the top of the page. After the UI is reloaded you should see the VAE dropdown menu at the top of the page:

In conclusion, VAEs enhance the visual quality of Stable Diffusion checkpoint models by improving image sharpness, color vividness, and the depiction of hands and faces. Downloading a VAE is an effective solution for addressing washed-out images. Incorporating VAEs into your workflow can lead to continuous improvement and better results. Explore the possibilities and elevate your Stable Diffusion experience with Variational Auto Encoders (VAEs).