How to Color Your Sketch Art with Stable Diffusion (A1111)

How to Color Your Sketch Art with Stable Diffusion (A1111)
Learn how to effortlessly color your sketch art using Stable Diffusion (A1111). This step-by-step guide will help you transform your sketches into vibrant artworks with ease.

1. Introduction

Welcome, fellow artists and creative souls!

Are you ready to bring your sketches to life with a burst of color? If you've been looking for a fun and easy way to transform your black-and-white sketches into vibrant, eye-catching masterpieces, then you're in the right place! Today, I'm thrilled to introduce you to the magic of Stable Diffusion (A1111). This amazing tool has completely revolutionized the way I color my sketches, and I can't wait to share all the tips and tricks I've learned with you.

Grab your favorite sketch, fire up Stable Diffusion (A1111), and let's dive into the colorful world of digital art together. Trust me, by the end of this guide, you'll be amazed at how effortlessly you can turn your sketches into stunning works of art. Let's get started and unleash our creativity!

To jumpstart our colorization process for sketch art, you'll need a sketch ready to go. In our last post, we delved into the thrilling process of turning an existing image into breathtaking sketch art! If you missed it, no worries—you can catch up by following the link below.

Transform Your Images into Sketch Art with Stable Diffusion (A1111)

With your initial sketch art piece ready, let's dive into the requirements for colorization with stable diffusion!

2. Requirements for Colorizing your Sketch Art

To start adding appealing colors to your lovely sketches or line drawings, it's important to set up ControlNet correctly with the pre-trained models, Lineart, and Shuffle.

If you haven't installed ControlNet yet, don't worry! We've put together a detailed guide that provides clear instructions on how to get it up and running. You can find the tutorial down below:

How to Install ControlNet Extension in Stable Diffusion (A1111)

Once you’ve installed ControlNet, it’s time to download the pre-trained models for "Lineart" and "Shuffle." Head over to the pre-trained models page on HuggingFace and download these specific models:

  • control_v11e_sd15_shuffle_fp16.safetensors
  • control_v11p_sd15_lineart_fp16.safetensors

Next, place these files in the following directory within the ControlNet extension: "extensions/sd-web-ui/ControlNet/models" inside your Stable Diffusion folder. Let's get those models set up so we can dive into the fun part – bringing your sketches to life with vibrant colors!

Stable Diffusion in the Cloud⚡️Run Automatic1111 in your browser in under 90 seconds
20% bonus on first deposit

3. Controlnet Settings (Txt2img)

Open Stable Diffusion and navigate to the txt2img tab. First, scroll down to the ControlNet panel. In the first ControlNet panel (0), paste your initial sketch art and apply the following settings:

Controlnet Unit 0 (Lineart)


  • Select "Enable" checkbox
  • Select "Pixel Perfect" checkbox
  • Control Type: Lineart
  • Preprocessor: lineart_standard (from white bg & black line)
  • Model: control_v11p_sd15_lineart_fp16.safetensors
  • Control Weight: 1 (default)
  • Control Mode: Balanced (default)

Controlnet Unit 1 (Shuffle)

Next, we'll open another ControlNet unit (1) and upload a color palette image to the canvas, using the settings listed below:


  • Select "Enable" checkbox
  • Select "Pixel Perfect" checkbox
  • Control Type: Shuffle
  • Preprocessor: shuffle
  • Model: control_v11e_sd15_shuffle [526bfdae]
  • Control Weight: 0.5 (default)
  • Control Mode: Balanced (default)

Note: If the colors are too bright in the final rendered image, try increasing the control weight.

4. Base Txt2img Settings

Scroll to the top, where we'll select the checkpoint for our image inference. We'll choose a checkpoint named "Photon." We'll leave the prompt empty, but feel free to add some keywords to highlight specific colors if you'd like! Since we've already provided a color palette image within the Shuffle ControlNet panel, leaving it empty works perfectly.

Below, you’ll find the basic settings we need to tweak before we click the magical "Generate" button and watch our black and white sketch burst into vibrant colors!


  • Checkpoint: Photon
  • Sampling Method: Euler a
  • Sampling Steps: 30
  • Width & Height: 888 x 496 (16:9 ratio, this should be the same as our intial sketch ratio)
  • CFG Scale: 7
  • Seed: -1

Once all the settings are configured, we can click on the "Generate" button!

Stable Diffusion in the Cloud⚡️Run Automatic1111 in your browser in under 90 seconds

5. Colored Sketch Art Examples

After the render completes, we've transformed our initial sketch art into a vibrant colorized piece using Stable Diffusion and ControlNet (LineArt & Shuffle). Below, you can see a collage featuring a variety of colorized sketches, each using different color palettes with the Shuffle ControlNet panel.


6. Conclusion

Congratulations! You've now mastered the art of effortlessly coloring your sketch art using Stable Diffusion (A1111). By following this step-by-step guide, you've learned how to transform your black-and-white sketches into vibrant, eye-catching artworks. With the power of ControlNet (LineArt & Shuffle), you can now explore endless possibilities in digital artistry, experimenting with different color palettes and settings to bring your creative visions to life.

Frequently Asked Questions

ControlNet, specifically LineArt and Shuffle modules, plays a crucial role in refining and customizing colorization outcomes. LineArt ensures accurate rendering of sketch details, while Shuffle allows for dynamic color palette adjustments. Together, they empower artists to achieve precise and personalized color results.

To achieve optimal color results, pay attention to the ControlNet settings, particularly the Control Weight parameter. Adjust this setting as needed to control the intensity and brightness of colors in your final artwork.

character creation tool card

AI Image Generation Tool

Create Your AI Images in Seconds!

News & Updates