Mastering Pose Changes: Stable Diffusion & ControlNet

Updated
Mastering Pose Changes: Stable Diffusion & ControlNet
Capture the essence of each pose as you transition effortlessly. Delve into the world of Stable Diffusion and ControlNet for seamless pose changes.

1. Introduction

Step into a world where fashion meets the future, powered by cutting-edge technology like Stable Diffusion. Join us on a journey that explores the art of changing poses while safeguarding every detail of your original image. Witness the seamless synergy of creativity and technology, shaping a new era of innovative style transformations!

2. Requirements for Pose Changing

Requirement 1: Stable Diffusion

To initiate a seamless journey through pose transformations, it's crucial to have Stable Diffusion installed as the foundation of our creative process. If you haven't set it up already, no need to worry. We've created a clear installation guide on how to install stable diffusion on windows.

Requirement 2: ControlNet & OpenPose Model

Upon successfully downloading and installing Stable diffusion, it is essential to also have ControlNet installed. We will use ControlNet for managing the posture of our fashion model. Additionally, downloading the OpenPose model is necessary. To assist you further, we provide an installation guide for ControlNet down below.

How to Install ControlNet Extension in Stable Diffusion (A1111)

Requirement 3: Initial Image

Ensure that you have an initial image prepared, or alternatively, you can generate one directly within the txt2img tab. If you have an image ready, we can move forward to the txt2img tab. To kickstart our creative process, we will start off with the following image below. how-to-change-poses-initial-image.webp

Stable Diffusion in the Cloud⚡️Run Automatic1111 in your browser in under 90 seconds
20% bonus on first deposit

3. Txt2img Settings

If you want to change the pose of an image you have created with Stable Diffusion then the process is simple. Simply drag the image in the PNG Info tab and hit “Send to txt2img”. This will copy over all the settings used to generate the image. how-to-change-poses-png-info-send-to-txt2img.webp

If you want to change the pose of a different image then adjust the settings in the Txt2img tab accordingly:

  • Prompts: Describe the details of the image and add keywords, alternatively you can drag the image in the img2img tab and press the “Interrogate CLIP” or the “Interrogate DeepBooru” button. I found that the DeepBooru method gives a better result. This will generate a prompt based on the image, this does need some finetuning. Don’t forget to add a negative prompt.
  • Sampling Method: This doesn’t matter too much, for photographs i recommend Euler a or DDIM. For more stylized images I recommend DPM++ 2M SDE Karras.
  • Sampling Steps: Set this to at least 30 or higher.
  • Width & Height: Use the same aspect ratio as your initial image. Don’t use too high values, a good maximum is somewhere between 768 and 1024.
  • CFG Scale: Use a balanced CFG scale of around 7,5.

Leave the rest of the settings untouched. how-to-change-poses-initial-image-settings.webp

4. ControlNet Settings

We will be utilizing 2 ControlNet models so make sure you have Multi ControlNet set to at least 2. This can be done in the Settings tab and then click ControlNet on the left. how-to-change-poses-multi-controlnet-setting.webp

ControlNet Unit 0

Firstly drag your initial image in the ControlNet Unit then change the following settings:

  • Control Type: Reference
  • Preprocessor: reference_only
  • Control Weight: Between 1 and 2, see what works best for you.
  • Control Mode: ControlNet is more important

Leave the rest of the settings at their default values. how-to-change-poses-controlnet-referenceonly.webp

ControlNet Unit 1

Drag your openpose image in the ControlNet unit, I have a pack with dynamic poses available on civitAI for free. If you’re unfamiliar with open pose, I recommend watching our openpose crash course on youtube.

  • Control Type: OpenPose
  • Preprocessor: None (Alternatively you can upload any image in the ControlNet unit and use the “openpose” preproccesor)
  • Model: Use the sd15 model
  • Starting Control Step: Use a value between 0 and 0.2.

Leave the rest of the settings at their default values. how-to-change-poses-controlnet-openpose.webp

Now make sure both ControlNet units are enabled and hit generate!

Stable Diffusion in the Cloud⚡️Run Automatic1111 in your browser in under 90 seconds

5. Examples

Example 1

how-to-change-poses-example-one.webp

Example 2

how-to-change-poses-example-two.webp

6. Conclusion

In conclusion, the fusion of fashion and cutting-edge technology has never been more captivating than with Stable Diffusion and ControlNet. As you've journeyed through this tutorial, you've gained insights into the transformative power of seamlessly changing poses while retaining the essence of your original images. By embracing these technologies, you've unlocked a realm of creative possibilities, where innovation and style converge in harmonious synergy. From requirements to settings, the tutorial has equipped you with the tools to navigate the intricacies of pose transformations, paving the way for a future where your images become a canvas for artistic expression. As you continue to experiment, refine, and master the art of pose changes, you're poised to reshape the landscape of visual storytelling with confidence and precision.

Frequently Asked Questions

Yes, Stable Diffusion and ControlNet are versatile tools that can be applied to various types of images, including photographs and stylized images. By adjusting settings you can achieve the desired pose change effect across different image styles.

You have the flexibility to use both your own images and those generated within the tutorial. If you've already created an image, ensure you copy the seed and settings for optimal results before proceeding to the txt2img tab.

While the tutorial primarily focuses on changing the pose of a single subject, you can apply similar principles to multiple subjects within the same image. Keep in mind that fine-tuning settings might be necessary to ensure accurate transformations.