How to Use Inpainting in Stable Diffusion: Beginner's Guide

Inpainting is a valuable technique for fixing small defects in images. It allows you to remove or replace specific areas in an image seamlessly. In this beginner's guide, we will explore step-by-step examples of using inpainting to fix image defects. Whether you're new to AI images or looking to enhance your inpainting skills, this guide will provide you with practical insights and techniques.

We will start with an original image and address specific issues using inpainting techniques. To begin this tutorial, we made the following original image using the txt2img tab in stable diffusion:

The image is not too bad, but there are some things that I would like to address.

• The facial features appear artificial and unnatural.
• I want a different house/villa on the left side of the picture

This is where the inpaint feature comes into play. Once you have generated the image using txt2img, you can choose to send the image to inpaint by clicking the "Send to inpaint" button below the generated image:

If you have another image that was not initially created using the txt2img feature, you can access the img2img tab. From there, select the 'inpaint' option and upload your image to initiate the process.

In order to inpaint specific areas, we need to create a mask using the AUTOMATIC1111 GUI. This mask will indicate the regions where the Stable Diffusion model should regenerate the image. The initial aspect we want to focus on is her face, therefore, we will create a mask using the paintbrush tool as illustrated below:

Once the mask has been created using the paintbrush tool, we can proceed to adjust the inpaint settings.

To achieve the desired inpainting results, various settings need to be adjusted. This section covers important settings such as prompt, image size, face restoration, mask content, denoising strength, and batch size.

• Prompt: To address defects, you can utilize the original prompt as a basis. This allows you to generate multiple images specifically focused on a particular area. If you want to focus on a specific detail, we recommend modifying the prompt to only address the changes you desire.
• Image Size: It is necessary to adjust the image size to match the dimensions of the original image (704 x 392 in this case). When using the "Send to inpaint" option this will automatically be the original image size.
• Face Restoration: If you are inpainting faces, you can enable the "Restore faces" option. In this case we are ticking the box "Restore faces".
• Masked Content: I would advise to choose the "original" if you want the result to be guided by the color and shape of the original content. "Original" is often used for inpainting faces when the general shape and anatomy are acceptable, but slight modifications are desired.
• Denoising Strength: The denoising strength parameter determines the extent of changes compared to the original image. A value of 0 indicates no change, while a value of 1 produces an inpainted result unrelated to the original image.
• Batch Size: To have multiple options to choose from, generate several images at once. Set the seed to -1 to ensure each image is unique.
• CFG Scale: Just like in text-to-image scenarios, the CFG (Classifier Free Guidance) scale is a parameter that determines the level of influence your prompt has on the model's output. When you set the CFG scale to 1, the prompt will be mostly disregarded, while cranking it up to 30 will result in strict adherence to the prompt. Generally, a value of 7 strikes a good balance between following the prompt and allowing for some creative freedom.

Below are the settings used to restore the face of the image:

Here, you will find some examples of inpainted images obtained using the configured settings. The results demonstrate the effectiveness of inpainting in fixing defects and improving the overall image quality.

Example 1:

Exmaple 2:

As observed, the facial features now appear significantly more natural. It's truly remarkable!

In addition to fixing defects, inpainting can also be used to add new objects to an image. This section guides you through the process of adding an amazing looking villa on the left side of the existing image. Just as before we need to inpaint a mask using the brush tool on the left side.

Creating a Mask:

Result:

Achieving successful inpainting results necessitates both patience and skill. Here are some key takeaways for utilizing inpainting effectively:

• Tackle one small area at a time.
• In most cases, keeping the masked content at "Original" and adjusting the denoising strength yields favorable outcomes.
• Experiment with different masked content options to determine the most suitable one for your specific case.

Inpainting is a powerful technique that allows you to fix small defects in images and even add new elements. By following this beginner's guide, you have learned how to use Stable Diffusion AI and the AUTOMATIC1111 GUI to perform inpainting with step-by-step examples. With practice and experimentation, you can refine your inpainting skills and achieve impressive results.