How to Change Backgrounds with Stable Diffusion (Inpaint Anything)

How to Change Backgrounds with Stable Diffusion (Inpaint Anything)
Learn to change backgrounds effortlessly with Stable Diffusion: in-painting and Inpaint-Anything methods explained.

1. Introduction

Changing backgrounds has never been easier or more exciting, especially if you lack advanced Photoshop skills. In this tutorial, we'll walk you through the process of seamlessly turning one image into another using Stable Diffusion. We'll explore two powerful techniques: in-painting and the awesome inpaint-anything extension. Additionally, we'll delve into fine-tuning to address small background details. Let's dive into the magical world of Automatic1111.

2. Stable Diffusion Inpainting

To begin, select a Stable Diffusion checkpoint. For this tutorial, we recommend choosing an in-painting version. I'll opt for "ReV Animated inpainting v1.2.1."

Selecting the In-Painting Tab

Navigate to the "Image to Image" tab and select the "In-Painting" tab.

Drag and drop your starting image

Drag and drop your image into the tab. I'll be using this image as my starting image. how-to-change-background-in-stable-diffusion-inpaint-anything-starting-image.webp

Creating the Mask

Now, the crucial step is creating a mask for what you want to change or retain. In my case, I'll only mask the person in this image, because that is the only thing I want to retain and it's easier to just mask out the subject rather than masking the whole background. how-to-change-background-in-stable-diffusion-inpaint-anything-starting-image-inpainted.webp

Writing the Prompt

Compose a prompt for the image. For this example, let's use "explosions" to make the picture more engaging. You can add a generic negative prompt to balance it.

Setting Parameters

Adjust the parameters as follows:

  • Mask blur: I recommend 4 or higher, see what is best for you.
  • Mask mode: Inpaint not masked
  • Sampling method: DPM++ 2M SDE Karras
  • Sampling steps: Use a minimum of 25, but higher is better.
  • Width & Height: Use the appropriate dimensions (e.g., 768x512 for landscape).
  • Denoising strength: 1

Keep other parameters as default. parameters-img2img-stable-diffusion-inpaint-anything.webp

Generating a New Background

Generate a new background, and voila! You've transformed your image into something captivating. how-to-change-background-in-stable-diffusion-inpaint-anything-starting-image-inpaint-method.webp Clearly, this approach has its imperfections; in my situation, it results in a minor yet noticeable border around my character. This is where the second technique becomes essential.

Stable Diffusion in the Cloud⚡️Run Automatic1111 in your browser in under 90 seconds
20% bonus on first deposit

3. The Superior Technique: Inpaint-Anything

Now, let's explore another way to change backgrounds using inpaint-anything, an incredible extension for Stable Diffusion.

Install Inpaint-Anything

If you don't have the Inpaint-Anything extension, you can install it from extensions tab.

  • Click on "Extensions"
  • Click on "Available"
  • Click on "Load from"
  • Search for "Inpaint Anything"
  • Click on "Install"


Following the installation, make sure to restart the UI.

Select and download a Model

After installing the extension and restarting the UI head to the “Inpaint Anything” tab and select a segment Model. I'll use “sam_vit_l_0b3195.pth” but feel free to try out any model. Press “Download model” after selecting your Segment Anything Model, when download is complete we can move on to the next step.


Drag and Drop the Image

Drag and drop your image. I’ll be using the same image as before to have a good comparison. how-to-change-background-in-stable-diffusion-inpaint-anything-starting-image.webp

4. Masking with Inpaint-Anything

Inpaint Anything makes masking a breeze. After you’ve dropped your image click on “Run Segment Anything”, this should take a few moments.

Mark each colored segment by simply clicking on the color to create a mask. Be precise, and avoid selecting areas that shouldn't be masked, I recommend zooming in a bit to avoid mistakes. Regularly check the mask's progress to ensure accuracy, every few clicks press on “Create Mask” to check your progress. If you masked a wrong area by accident you can revert this by clicking on the back button in the segmented image.

Now you should have a very detailed mask. However I do recommend clicking on “Expand mask region” once or twice to expand the mask by 1 or 2 pixels. how-to-change-background-stable-diffusion-inpaint-anything-segmented.webp

Using the Mask

On the left side select the “Mask only” tab and click on “Get mask”. Then right under the mask hit “Send to img2img inpaint”. how-to-change-background-stable-diffusion-inpaint-anything-get-mask.webp

Now your highly detailed mask is in the img2img Inpaint upload tab. From here we can follow the same steps as before and compare our results.

Write your prompts, select the correct mask mode and dimensions. Adjust the sampling method and sampling steps and set the denoising strength to 1. I will use the same prompt as before to compare our results.

Stable Diffusion in the Cloud⚡️Run Automatic1111 in your browser in under 90 seconds

5. Results and Comparison

Inpaint-Anything offers more precision by allowing detailed segmentation of the image, reducing errors and delivering better results, as demonstrated in the comparison below. how-to-change-background-in-stable-diffusion-inpaint-anything-comparison.webp

6. Conclusion

You've just unlocked the power of Stable Diffusion for background transformations. Whether you're turning a simple picture into an action movie scene or fixing imperfections in your images, Stable Diffusion empowers you to create stunning visuals. Experiment, refine, and elevate your image editing game.

Frequently Asked Questions

Inpaint-Anything offers more precision by allowing detailed segmentation of the image. This reduces errors and can result in better, more accurate transformations. As shown in the comparison in the blog, it can be particularly useful when you need finer control over masking and background changes.

Yes, Stable Diffusion can be used for professional image editing tasks, including photography and graphic design. It offers advanced features and can help achieve high-quality results. However, like any tool, it may require practice to master for professional-grade work.

Some limitations of Stable Diffusion include the need for appropriate input images, potential artifacts in the generated results, and the requirement for a powerful computer for optimal performance. It's essential to experiment and refine your techniques to mitigate these issues.