How to Use Inpainting in Stable Diffusion: Beginner's Guide

Updated
How to Use Inpainting in Stable Diffusion: Beginner's Guide
Learn how to effectively utilize inpainting in Stable Diffusion with this comprehensive beginner's guide. Discover step-by-step instructions and techniques for achieving seamless image fixes. Master the art of inpainting and enhance your image editing skills.

1. Introduction

Inpainting is a valuable technique for fixing small defects in images. It allows you to remove or replace specific areas in an image seamlessly. In this beginner's guide, we will explore step-by-step examples of using inpainting to fix image defects. Whether you're new to AI images or looking to enhance your inpainting skills, this guide will provide you with practical insights and techniques.

2. Generating Base/Original Image (txt2img)

We will start with an original image and address specific issues using inpainting techniques. To begin this tutorial, we made the following original image using the txt2img tab in stable diffusion: how-to-use-inpainting-in-stable-diffusion-beginners-guide.png

The image is not too bad, but there are some things that I would like to address.

  • The facial features appear artificial and unnatural.
  • I want a different house/villa on the left side of the picture

This is where the inpaint feature comes into play. Once you have generated the image using txt2img, you can choose to send the image to inpaint by clicking the "Send to inpaint" button below the generated image: how-to-use-inpainting-in-stable-diffusion-beginners-guide_nextdiffusion_1.png

If you have another image that was not initially created using the txt2img feature, you can access the img2img tab. From there, select the 'inpaint' option and upload your image to initiate the process.

Stable Diffusion in the Cloud⚡️Run Automatic1111 in your browser in under 90 seconds
20% bonus on first deposit

3. Creating an Inpaint Mask

In order to inpaint specific areas, we need to create a mask using the AUTOMATIC1111 GUI. This mask will indicate the regions where the Stable Diffusion model should regenerate the image. The initial aspect we want to focus on is her face, therefore, we will create a mask using the paintbrush tool as illustrated below: how-to-use-inpainting-in-stable-diffusion-beginners-guide_nextdiffusion_2.png

Once the mask has been created using the paintbrush tool, we can proceed to adjust the inpaint settings.

4. Settings for Inpainting

To achieve the desired inpainting results, various settings need to be adjusted. This section covers important settings such as prompt, image size, face restoration, mask content, denoising strength, and batch size.

  • Prompt: To address defects, you can utilize the original prompt as a basis. This allows you to generate multiple images specifically focused on a particular area. If you want to focus on a specific detail, we recommend modifying the prompt to only address the changes you desire.
  • Image Size: It is necessary to adjust the image size to match the dimensions of the original image (704 x 392 in this case). When using the "Send to inpaint" option this will automatically be the original image size.
  • Face Restoration: If you are inpainting faces, you can enable the "Restore faces" option. In this case we are ticking the box "Restore faces".
  • Masked Content: I would advise to choose the "original" if you want the result to be guided by the color and shape of the original content. "Original" is often used for inpainting faces when the general shape and anatomy are acceptable, but slight modifications are desired.
  • Denoising Strength: The denoising strength parameter determines the extent of changes compared to the original image. A value of 0 indicates no change, while a value of 1 produces an inpainted result unrelated to the original image.
  • Batch Size: To have multiple options to choose from, generate several images at once. Set the seed to -1 to ensure each image is unique.
  • CFG Scale: Just like in text-to-image scenarios, the CFG (Classifier Free Guidance) scale is a parameter that determines the level of influence your prompt has on the model's output. When you set the CFG scale to 1, the prompt will be mostly disregarded, while cranking it up to 30 will result in strict adherence to the prompt. Generally, a value of 7 strikes a good balance between following the prompt and allowing for some creative freedom.

Below are the settings used to restore the face of the image: how-to-use-inpainting-in-stable-diffusion-beginners-guide_nextdiffusion_3.png

Stable Diffusion in the Cloud⚡️Run Automatic1111 in your browser in under 90 seconds

5. Inpainting Results

Here, you will find some examples of inpainted images obtained using the configured settings. The results demonstrate the effectiveness of inpainting in fixing defects and improving the overall image quality.

Example 1: how-to-use-inpainting-in-stable-diffusion-beginners-guide_nextdiffusion_4.png

Exmaple 2: how-to-use-inpainting-in-stable-diffusion-beginners-guide_nextdiffusion_5.png

As observed, the facial features now appear significantly more natural. It's truly remarkable!

7. Inpaint New Objects to an Image

In addition to fixing defects, inpainting can also be used to add new objects to an image. This section guides you through the process of adding an amazing looking villa on the left side of the existing image. Just as before we need to inpaint a mask using the brush tool on the left side.

Creating a Mask: how-to-use-inpainting-in-stable-diffusion-beginners-guide_nextdiffusion_6.png

Result: how-to-use-inpainting-in-stable-diffusion-beginners-guide_nextdiffusion_9.png

Stable Diffusion in the Cloud⚡️Run Automatic1111 in your browser in under 90 seconds

8. Tips for Inpainting in Stable Diffusion

Achieving successful inpainting results necessitates both patience and skill. Here are some key takeaways for utilizing inpainting effectively:

  • Tackle one small area at a time.
  • In most cases, keeping the masked content at "Original" and adjusting the denoising strength yields favorable outcomes.
  • Experiment with different masked content options to determine the most suitable one for your specific case.

9. Conclusion

Inpainting is a powerful technique that allows you to fix small defects in images and even add new elements. By following this beginner's guide, you have learned how to use Stable Diffusion AI and the AUTOMATIC1111 GUI to perform inpainting with step-by-step examples. With practice and experimentation, you can refine your inpainting skills and achieve impressive results.