Vibrant Sketch Colorization: Stable Diffusion & ControlNet

Updated
Vibrant Sketch Colorization: Stable Diffusion & ControlNet
Elevate sketches with precision using Stable Diffusion & ControlNet for stunning colorization. Transform monochrome drawings into vibrant masterpieces.

1. Introduction

Step into a realm where sketches come alive in brilliant color. Discover how we use Stable Diffusion and ControlNet to transform plain drawings into vibrant artworks. Let's explore the magic of blending technology and art to make static sketches tell vivid visual stories.

2. Requirements: Sketch Colorization

To start adding appealing colors to your lovely sketches or line drawings, it's important to set up ControlNet correctly with the pre-trained models, Lineart, and Shuffle. If you haven't installed ControlNet yet, don't worry, we've put together a detailed guide that provides clear instructions on how to get it up and running. You can find the tutorial down below:

How to Install ControlNet Extension in Stable Diffusion (A1111)

Once ControlNet is installed correctly, make sure you have a captivating sketch or a carefully crafted line drawing that you're ready to infuse with color. If you haven't prepared one yet, no need to be concerned – we've a guide that walks you through the process of creating sketch art using Stable Diffusion

Stable Diffusion in the Cloud⚡️Run Automatic1111 in your browser in under 90 seconds
20% bonus on first deposit

3. Txt2img Settings (Stable Diffusion)

In the "txt2img" section, we'll choose for the revAnimated checkpoint. When it comes to the sampling method, we'll select DDIM and set the sampling steps to 30. For the dimensions of the output image, we'll make sure to resize it to match your sketch's dimensions. In our case, this translates to a resolution of 6:19, a width of 400 pixels, and a height of 712 pixels.

If the need arises to upscale the image, rest assured that we can easily utilize the Ultimate Upscale SD script for that purpose.

sketch-colorization-using-stable-diffusion-controlnet-txt2img-settings-model-checkpoint.webp

  • Sampling Method: DDIM
  • Sampling Steps: 30
  • Width and Height: Resize to match the size of your sketch image; for us, it's a resolution of 6:19 with a width of 392 and a height of 704.
  • CFG: 7
  • Seed: -1

4. Customizing ControlNet Settings

ControlNet Unit 0

For the first ControlNet configuration, place your prepared sketch or line art onto the canvas through a simple drag-and-drop action. Next, let's move forward by adjusting the following settings listed down below:

sketch-colorization-using-stable-diffusion-controlnet-controlnet-settings-sketch-line-art-image.webp

  • Enable ControlNet
  • Enable Pixel Perfect
  • Control Type: Lineart
  • Control Weight: 2 (to retain visible lines in the colorization output)
  • Control Mode: ControlNet is more important

Note: In place of selecting "lineart" as the control type, you also have the alternative of opting for "Canny" as the control type.

ControlNet Unit 1

For the second ControlNet unit, we'll introduce a colorized image that represents the color palette we intend to apply to our initial sketch art. Below, you'll find the settings required for this process:

sketch-colorization-using-stable-diffusion-controlnet-unit1-controlnet-settings-color-palette-colored-image.webp

  • Enable ControlNet
  • Enable Pixel Perfect
  • Enable Allow Preview
  • Control Type: Shuffle
  • Control Weight: 1
  • Control Mode: Balanced

With all the elements aligned, it's time to click the "Generate" button and witness the transformation of your incredible sketch art into a vibrant masterpiece, adorned with stunning colors.

Stable Diffusion in the Cloud⚡️Run Automatic1111 in your browser in under 90 seconds

5. Example: Colorized Sketch / Line art

Once ControlNet has completed the rendering process for your image, you'll have the opportunity to observe the remarkable results it produces. Feel free to explore and experiment with various color palettes to achieve the desired appearance for your sketch or line art.

sketch-colorization-using-stable-diffusion-controlnet-unit1-controlnet-settings-color-palette-colored-image-colorized-image-output-example.webp

6. Conclusion

our sketches have blossomed from mere lines into vibrant tales, all thanks to Stable Diffusion and ControlNet. As you experiment and click "Generate," remember that your artistic exploration has just begun. Unleash your creativity, and let your sketches tell endless stories in vibrant color.

We're excited to have embarked on this journey with you, and we eagerly anticipate welcoming you to our next tutorial. Until then, keep exploring, keep creating, and keep infusing life into your art. Your creative potential knows no bounds.

Frequently Asked Questions

Stable Diffusion and ControlNet are a dynamic duo that takes your sketches from monochrome to mesmerizing. Stable Diffusion ensures seamless color transitions, while ControlNet grants you control over the colorization process. Together, they breathe life into your sketches, transforming them into vivid and expressive works of art.

You have the creative reins! In the "txt2img" tab, select your preferred settings, including the checkpoint, sampling method, and output image dimensions. ControlNet settings enable you to fine-tune the level of color influence and even retain original lines if desired. Experiment with different palettes and settings to achieve your unique vision. Click "Generate," and watch your sketches burst into a kaleidoscope of colors.