3 Powerful Techniques to Control Lighting in Stable Diffusion

Lighting is an indispensable aspect of photography that profoundly influences the quality and atmosphere of an image. Its impact extends beyond mere illumination, allowing you to elevate your subjects, infuse depth and dimension, evoke emotions, and emphasize critical elements.

In this blog post, we will delve into the realm of lighting control and explore three effective methods that enable you to wield mastery over lighting:

• Lighting Keywords
• Regional Prompting
• ControlNet img2img

Lighting is a fundamental aspect of photography, capable of transforming an image's quality, mood, and impact. Understanding how to effectively control lighting is essential for photographers seeking to create visually stunning and captivating compositions.

Adding Keywords to the Prompt:

Keywords serve as the key to unlocking desired lighting effects in Stable Diffusion photography. Learn how to strategically incorporate specific keywords into your prompts to control lighting and enhance the overall composition. We will guide you through the process of selecting and utilizing the right keywords to achieve various lighting effects, providing a solid foundation for your creative endeavors.

Let's begin and generate some images without any lighting-related keywords, to observe the impact that incorporating such keywords can have. You can see the starting image below:

Example 1 (No lighting keywords):

Example 2 (No lighting keywords):

The above images were generated by using these settings:

• Positive Prompt: best quality, ultra high res, (photorealistic:1.4), 1girl, (asian girl:1.3), off-shoulder black shirt, skirt, black choker, (full body portrait:1.3), (faded ash gray hair:1.4), slim, (medium breasts:1), looking at viewer, outside, (cleavage:1.2)
• Negative Prompt: (worst quality:1.2), (low quality:1.2), (lowres:1.1), (monochrome:1.1), (greyscale), multiple views, comic, sketch, (((bad anatomy))), (((deformed))), (((disfigured))), watermark, multiple_views, mutation hands, mutation fingers, extra fingers, missing fingers, watermark,
• Sampling Steps: 30
• Sampler: DDIM
• CFG scale: 8
• Size: 768x432
• Model: ICBINP
• Clip Skip: 2

Now let's explore some examples where we will apply some lightning keywords.

Example of Different Lighting Effects:

Discover a plethora of lighting effects that can be achieved by using the right combination of keywords. Let's explore some examples:

• Volumetric lighting, Atmospheric illumination, God rays: These lighting keywords refer to the presence of distinct light beams within an image, employed in photography to enhance the perception of depth and volume. When adding these keywords to your prompt you will get the following effect:

Witness the transformative power of the lighting keywords and gain inspiration for incorporating them into your own photographic projects. For a full list of lightning keywords you can use our free to use Stable Diffusion Prompt Generator. This tool highlights different lightning keywords, so feel free to experiment with these prompt outputs.

Tips for Using Lighting Keywords:

Unlock the full potential of lighting keywords with our expert tips and recommendations:

• Give greater importance to the keyword if the desired effect is not evident. For example use (god rays:1.5) instead of just "god rays".
• Note that not all lighting keywords yield consistent results; try generating a few images simultaneously for testing purposes.
• Discover additional lighting keywords by utilizing our Free Prompt Generator.

When using lighting keywords in the prompt, the effects typically apply to the entire image, influencing the overall lighting composition. However, if you desire more precise control over the lighting in specific areas, you can utilize the Regional Prompter extension. If you have not yet installed the Regional Prompter extension you can find a comprehensive guide below:

Regional Prompter Extension: Image Composition in Stable Diffusion

How to use the Regional Prompter in Stable Diffusion?

1. Unfold the Regional Prompter section on the txt2img page.
2. Activate the Regional Prompter
3. Select the following options in the Regional Prompter

• Active: Yes
• Use common prompt: Yes
• Split mode: Vertical
• Divide Ratio: 1, 1

4. Click "visualize and make template". You will see the region image below indicating two regions: region 0 on the top and region 1 on the bottom divided equally in a 1-to-1 ratio.

5. Now it is time to create our prompt: Keep in mind that the prompts are separated by the keyword BREAK.

Lets explain the following prompt:

We have three prompts above seperated by a BREAK

• Common prompt = Portrait image of an asian women
• Prompt for region 0 = (god rays:1.2), (volumetric lighting:1.2)
• Prompt for region 1 = (dimly lit:1.4)

Example Output & Explanation:

So the image is divided in the following manner by the Regional Prompt:

With the Regional Prompter extension, you gain the ability to create distinct visual and lighting effects in different regions of your image. This extension empowers you to customize and control the lighting composition, allowing for unique and captivating visuals throughout various areas of your photograph. The following article explains Regional Prompting in more detail: Regional Prompter Extension: Image Composition in Stable Diffusion.

Tips for Regional Prompting

• Adjust keyword weights for desired effects. Instead of just adding "dimly lit" you can add (dimly lit: 1.4).
• Regional prompting may require multiple iterations for optimal results.

ControlNet offers a precise method for controlling illumination in your photography. To utilize this advanced technique, you will need to have the ControlNet extension installed. If you haven't installed it yet, follow the provided tutorial for step-by-step instructions on installing the ControlNet extension.

Once installed, you will unlock the ability to achieve precise control over the lighting in your images, allowing you to create stunning and visually captivating compositions.

ControlNet Settings (Img2Img):

1. On the img2img page, navigate to the ControlNet section.
2. Upload an Portrait or Photography to ControlNet Unit 0.
3. Use the following ControlNet settings:

• Enable: Yes
• Pixel Perfect: Yes
• Allow preview: Yes
• Control Type: Depth
• Preprocessor: depth_zoe
• Model: control_depth-fp16
• Control Weight: 0.6

The uploaded portrait image was created with txt2img:

• Positive Prompt: best quality, ultra high res, (photorealistic:1.4), 1girl, (asian girl:1.3), off-shoulder black shirt, skirt, black choker, (full body portrait:1.3), (faded ash gray hair:1.4), slim, (medium breasts:1), looking at viewer, inside, (cleavage:1.2)
• Negative Prompt: (worst quality:1.2), (low quality:1.2), (lowres:1.1), (monochrome:1.1), (greyscale), multiple views, comic, sketch, (((bad anatomy))), (((deformed))), (((disfigured))), watermark, multiple_views, mutation hands, mutation fingers, extra fingers, missing fingers, watermark,
• Sampling Steps: 40
• Sampler: DDIM
• CFG scale: 8
• Size: 768x432
• Model: ICBINP
• Clip Skip: 2

After generating an image with txt2img you can send it to img2img so that the prompt and dimensions will be replicated inside the img2img tab:

4. img2img canvas settings

• Upload a lightning image to the img2imag canvas (so replace the original image which appeared when you clicked the "send to img2img" button from the txt2img tab).

You can find these images just by googling something like: "spotlight" or "god rays".

• Set the resize mode to "Just Resize".

• Set denoising strength to 0.95.

• Press the "Generate" button.

You should get images with the light source behind the girl. See examples below:

Mastering the art of controlling lighting in Stable Diffusion photography is a transformative skill that can elevate your images to new heights. Throughout this blog post, we have explored three powerful techniques that provide you with precise control over lighting: using lighting keywords, regional prompting, and harnessing the capabilities of ControlNet.

By mastering these three techniques, you gain the power to create captivating and visually striking images that convey your artistic vision. Whether you want to enhance the subject, evoke emotions, or create unique lighting compositions, these techniques provide you with the tools to unlock your creative potential.