Changing backgrounds has never been easier or more exciting, especially if you lack advanced Photoshop skills. In this tutorial, we'll walk you through the process of seamlessly turning one image into another using Stable Diffusion. We'll explore two powerful techniques: in-painting and the awesome inpaint-anything extension. Additionally, we'll delve into fine-tuning to address small background details. Let's dive into the magical world of Automatic1111.
To begin, select a Stable Diffusion checkpoint. For this tutorial, we recommend choosing an in-painting version. I'll opt for "ReV Animated inpainting v1.2.1."
Navigate to the "Image to Image" tab and select the "In-Painting" tab.
Drag and drop your image into the tab. I'll be using this image as my starting image.
Now, the crucial step is creating a mask for what you want to change or retain. In my case, I'll only mask the person in this image, because that is the only thing I want to retain and it's easier to just mask out the subject rather than masking the whole background.
Compose a prompt for the image. For this example, let's use "explosions" to make the picture more engaging. You can add a generic negative prompt to balance it.
Adjust the parameters as follows:
Keep other parameters as default.
Generate a new background, and voila! You've transformed your image into something captivating. Clearly, this approach has its imperfections; in my situation, it results in a minor yet noticeable border around my character. This is where the second technique becomes essential.
Now, let's explore another way to change backgrounds using inpaint-anything, an incredible extension for Stable Diffusion.
If you don't have the Inpaint-Anything extension, you can install it from extensions tab.
Following the installation, make sure to restart the UI.
After installing the extension and restarting the UI head to the “Inpaint Anything” tab and select a segment Model. I'll use “sam_vit_l_0b3195.pth” but feel free to try out any model. Press “Download model” after selecting your Segment Anything Model, when download is complete we can move on to the next step.
Drag and drop your image. I’ll be using the same image as before to have a good comparison.
Inpaint Anything makes masking a breeze. After you’ve dropped your image click on “Run Segment Anything”, this should take a few moments.
Mark each colored segment by simply clicking on the color to create a mask. Be precise, and avoid selecting areas that shouldn't be masked, I recommend zooming in a bit to avoid mistakes. Regularly check the mask's progress to ensure accuracy, every few clicks press on “Create Mask” to check your progress. If you masked a wrong area by accident you can revert this by clicking on the back button in the segmented image.
Now you should have a very detailed mask. However I do recommend clicking on “Expand mask region” once or twice to expand the mask by 1 or 2 pixels.
On the left side select the “Mask only” tab and click on “Get mask”. Then right under the mask hit “Send to img2img inpaint”.
Now your highly detailed mask is in the img2img Inpaint upload tab. From here we can follow the same steps as before and compare our results.
Write your prompts, select the correct mask mode and dimensions. Adjust the sampling method and sampling steps and set the denoising strength to 1. I will use the same prompt as before to compare our results.
Inpaint-Anything offers more precision by allowing detailed segmentation of the image, reducing errors and delivering better results, as demonstrated in the comparison below.
You've just unlocked the power of Stable Diffusion for background transformations. Whether you're turning a simple picture into an action movie scene or fixing imperfections in your images, Stable Diffusion empowers you to create stunning visuals. Experiment, refine, and elevate your image editing game.
Inpaint-Anything offers more precision by allowing detailed segmentation of the image. This reduces errors and can result in better, more accurate transformations. As shown in the comparison in the blog, it can be particularly useful when you need finer control over masking and background changes.
Yes, Stable Diffusion can be used for professional image editing tasks, including photography and graphic design. It offers advanced features and can help achieve high-quality results. However, like any tool, it may require practice to master for professional-grade work.
Some limitations of Stable Diffusion include the need for appropriate input images, potential artifacts in the generated results, and the requirement for a powerful computer for optimal performance. It's essential to experiment and refine your techniques to mitigate these issues.