Regional Prompter Extension: Image Composition in Stable Diffusion

Join us on a mesmerizing journey of creativity as we delve into the realm of stable diffusion and regional prompter. Follow along as we demonstrate the process step-by-step, highlighting the various settings and techniques to unlock your imagination's full potential. Get ready to witness your wildest visions come to life in this captivating blog post!

Regional Prompter is a powerful extension that opens up a world of possibilities for image composition in Stable Diffusion. With Regional Prompter, you can blend different prompts to create stunning images that combine multiple elements seamlessly. Whether you want to merge contrasting themes or mix diverse subjects into a single creation, Regional Prompter provides the tools to bring your imagination to life. By understanding the various settings and techniques of Regional Prompter, you can harness its full potential and unlock a new level of creativity.

To begin your journey with Regional Prompter, you'll need to install the extension on Stable Diffusion. The installation process is straightforward and can be completed in a few simple steps:

• Navigate to the "extensions" menu in Stable Diffusion.

• Look for the "Available" section and click on "Load From."

• In the search bar, type "Regional Prompter" and select the extension from the results.

• Click on the "Install" button to initiate the installation process.

• Once the installation is complete, go to the "Install" tab and locate Regional Prompter in the list of installed extensions.

• Click on "Check for updates" and then on "Apply and restart UI" to restart the Stable Diffusion user interface.

Congratulations! You now have Regional Prompter successfully installed on Stable Diffusion, ready to explore the endless creative possibilities it offers. The Region Prompter expansion panel should now be visible when you scroll down in the "txt2img" tab.

Allow me to present a straightforward example why we want to use the "Regional Prompter". Suppose you desire to create an image featuring a man with blonde hair and a woman with red hair. For this purpose, we will employ a basic prompt:

Prompt: a man with blonde hair, a woman with red hair

At times, you may receive the desired outcome as described, but frequently, Stable Diffusion encounters confusion regarding the appropriate pairing of hair colors with individuals. This simple prompt will sometimes even generate two women:

What led to this situation? Why does Stable Diffusion struggle with such a seemingly simple task? The self-attention mechanism mistakenly associates the hair color with the wrong person.

To address this issue, a solution exists: explicitly indicate the prompt for a man with blonde hair on the left-hand side of the image and the woman with red hair on the right-hand side of the image. This is where the "Regional Prompter" comes to the rescue.

Using the Regional Prompter:

• Unfold the Regional Prompter section on the txt2img page.
• Check "Active" to activate the regional prompter.

Settings:

• Generation mode: Attention
• Divide mode: Horizontal
• Divide Ratio: 1, 1

Visualize Template:

• Click "visualize and make template".

Above, you will observe an image divided into two regions: region 0 on the left and region 1 on the right, each occupying an equal 1-to-1 ratio.

Now change the initial prompt to this: a man and a woman, a man with blonde hair BREAK a man and a woman, a woman with red hair

The prompts are separated by the keyword BREAK. We have two prompts above:

• Prompt for region 0 (left) = a man and a woman, a man with blonde hair
• Prompt for region 1 (right) = a man and a woman, a woman with red hair

Now click "Generate" to see the result:

Stable Diffusion consistently produces a depiction of a man with blonde hair in region 0 (left) and a woman with red hair in region 1 (right). However, it is important to note that this outcome is not guaranteed in every instance. It achieves successful results approximately 80% of the time, which is still quite impressive!

Using Common prompt:

You may have noticed the two prompts shared a common part “a man and a woman”.

Prompt: a man and a woman, a man with blonde hair BREAK a man and a woman, a woman with red hair

If you don’t have the common prompt, Stable Diffusion will only generate one person The prompt below is the prompt without a common prompt and this will generate one person:

Prompt: a man with blonde hair BREAK a woman with red hair

The issue arises from the fact that both prompts for the left and right regions refer to a single individual, leading to the generation of only one person in the resulting image.

To address this, it is essential to explicitly communicate to Stable Diffusion that the image represents two distinct individuals -- a man and a woman.

That’s why you need a common prompt for “a man and a woman”.

Lets Generate the correct image with a common prompt:

1. Check the option "Use common prompt" in the regional prompter extension.
2. Now you can add the common prompt "a man and a woman" at the beginning.

Prompt: a man and a woman BREAK a man with blonde hair BREAK a woman with red hair

We have three prompts above (Seperated by the BREAK word)

• Common prompt = a man and a woman
• Prompt for region 0 = a man with blonde hair
• Prompt for region 1 = a woman with red hair

The common prompt is added to the beginning of the prompt for each region.

Example 1:

Example 2:

The interface provides a range of options and settings to customize your prompts and achieve the desired results. By familiarizing yourself with these elements, you can navigate the interface with confidence and create captivating images.

Exploring Split Mode and Divide Ratio:

In the Regional Prompter interface, two crucial aspects that shape the composition of your images are the split mode and divide ratio.

• Split mode: The split mode determines the orientation of the split: horizontal or vertical. Choosing the appropriate split mode is essential for achieving the desired visual impact.
• Divide Ratio: Allows you to control how the image is divided into sections and the influence each section has on the final result.

Horizontal Split

Horizontal split divides the image into left and right sections, while vertical split divides it in top and bottom sections.

To divide the regions in left and right, select horizontal in divide mode. Each region is represented by a number separated by commas. The number represents the size of the region.

Divide Ratio (1,1,1)

Divide Ratio (1,2,1)

Vertical Split

Vertical split divides the image into top and bottom sections.

Divide Ratio (1,1,1)

Divide Ratio (2,1,2)

Generation Modes: Attention vs. Latent:

Regional Prompter offers two distinct generation modes: attention and latent. These modes determine how the model focuses on the prompts and influences the image creation process. Experimenting with both modes can yield different results, so it's worth exploring and comparing the outputs to find the mode that aligns best with your creative vision.

Base Ratio: Controlling Prompt Influence:

The base ratio setting in Regional Prompter enables you to control the influence of the base prompt over the entire image. By adjusting this ratio, you can fine-tune the prominence of the base prompt and the extent to which the subsequent sections impact the final composition. Finding the right balance with the base ratio allows you to achieve the desired blend of different prompts.

Creating Sections in the Prompt:

Regional Prompter enables you to create sections within the prompt, allowing you to assign specific prompts to different areas of the image. By defining these sections, you can seamlessly blend different themes, subjects, or styles into a cohesive composition. To differentiate between the sections, the "BREAK" keyword is used. Placing the BREAK keyword in the prompt helps Regional Prompter identify the boundaries and allocate different prompts to distinct areas of the image.

2D regions in Regional Prompter:

You can divide the region both vertically and horizontally in an image. The rules are:

• The rows are separated by semicolons (;).
• Each row consists of a sequence of numbers separated by commas, such as 1,1,1.
• In each row, the first number indicates the height of that specific row, while the following numbers indicate the widths of the corresponding regions.

2D Example:

Divide Ratio (1,1,1;2,1,1):

• This defines two rows.
• The height of the first row is 1. The height of the second row is 2.
• Each row has two regions with equal widths (1,1).
• There are 4 regions in total.

Let's say I create an image with the following prompt:

Positive Prompt: ((best quality)), ((masterpiece)), ((detailed)), ((Intricate)), (Breathtaking), a witch, highly detailed face, half body, studio lighting, dramatic lighting, highly detailed clothing, looking at you, mysterious, dramatic lighting, (full moon:1.3)

Negative Prompt: ng_deepnegative_v1_75t, bad-artist, bad-hands-5, (worst quality, low quality:1.4), multiple views, multiple panels, blurry, watermark, letterbox, text, character, human, people, 1boy, signature, medium quality, deleted, lowres, cleavage, frame, watermark, signature, watermark, signature, nude, nsfw

We get some decent examples:

It is satisfactory, but it is impossible to manipulate the positions of the moon. Your only option is to repeatedly click the Generate button until you achieve the desired arrangements. This is where the regional prompter can help.

Use the following settings:

• Divide mode: Horizontal
• Use common prompt: Yes
• Divide ratio: 1,1,1;2,1,1

Our goal is to put the moon in the left top corner (Region 0). So let's change the prompt:

Prompt: ((best quality)), ((masterpiece)), ((detailed)), ((Intricate)), (Breathtaking), a witch, highly detailed face, half body, studio lighting, dramatic lighting, highly detailed clothing, looking at you, mysterious, dramatic lighting BREAK (full moon:1.3) BREAK BREAK BREAK

The first part of the prompt is our "Common prompt" where we removed the "full moon:1.3" part because we want to specify this in region 0 after the BREAK. After changing the prompt click "Generate".

Examples:

In summary, the addition of Regional Prompter to Stable Diffusion brings exciting possibilities for image composition. With Regional Prompter, you can blend different prompts effortlessly, resulting in stunning images that seamlessly combine multiple elements. Whether you want to mix contrasting themes or merge diverse subjects into a single creation, Regional Prompter provides the necessary tools to bring your creative ideas to life. By understanding its settings and techniques, you can fully explore the potential of Regional Prompter and unlock a new level of artistic expression.