Create Hyper-Realistic AI Influencers (Stable Diffusion & Roop)

Welcome to today's exploration of the fascinating world of crafting hyper-realistic AI influencers. In this journey, we'll delve into the power of Stable Diffusion and Roop, starting by crafting our initial face portrait as the foundation. Then, we'll harness the capabilities of the Roop Extension to generate consistent AI influencer faces. To add even more versatility, we'll employ ControlNet and OpenPose to transform our AI influencer into a myriad of captivating poses. Let's embark on this exciting adventure of AI influencer creation!

Before we begin making our AI Influencer, here's what we need to start:

ControlNet Extension & OpenPose model

To get started, it's essential to have the ControlNet Extension installed. Additionally, having the OpenPose model installed is crucial. Watch and learn how to install the ControlNet Extension.

Roop Extension

If you don't have the Roop extension installed, you can follow our step-by-step written tutorial.

How to Face Swap in Stable Diffusion with Roop Extension

Now with everything in place it is time to kick things off. Let's create our face portrait of our fake AI Influencer. We will eventually provide this portrait to the Roop Extension to get a consistant faces when creating our posts for Instagram, Tiktok and any other social media platform.

Some things to keep in mind when generating the face portrait:

• Dimensions of the image: We will use a dimension of 512x512, which is perfect for a face portrait.

• Your Checkpoint/Model: We aim for a hyper realistic portrait so we will opt for the RealisticVision Checkpoint.

• Your Prompt: The prompt will determine the look of your AI Influencer, so make sure you take the time to craft the perfect prompt which resembles your desired face. Also make sure the portrait is facing the camera (This way Roop can face swap much better when activated). For this tutorial we will create a beautiful woman with red hair as our AI influencer. We generated several 512x512 portraits to choose from:

• Positive Prompt: best quality, beautiful, 30 year old woman, ginger hair, (full face portrait:1.2), face portrait, detailed face, freckles, hyper realistic, (facing camera), (looking into the camera), studio portrait, natural lightning, perfect shadows, (white background:1.1)
• Negative Prompt: (worst quality:1.2), (low quality:1.2), (lowres:1.1), (monochrome:1.1), (greyscale), multiple views, comic, sketch, (((bad anatomy))), (((deformed))), (((disfigured))), mutation hands, mutation fingers, extra fingers, missing fingers, watermark
• Sampling Steps: 30
• Sampler: DDIM
• CFG scale: 7
• Size: 512x512
• Model: RealisticVision

After generating our AI portraits, it's time to choose your preferred facial portrait and proceed to the next step. In the upcoming phase, we will use this selected facial portrait as input for the Roop Extension to generate images featuring that particular face. Let's take a closer look.

To activate Roop, simply scroll down within the "txt2img" tab and access the "Roop" expansion panel. Within this panel, you can upload the portrait you've chosen as your favorite from your creations. Don't forget to check the "Enable" box to ensure that the Roop Extension is turned on:

Ensure that the selected portrait features a fully visible face with no obstruction from hair.

Prompts

Now, let's try generating some images with the Roop Extension turned on. It's crucial to understand that while the Roop Extension will replace the face in the generated image with your chosen portrait, it typically doesn't alter the eye color and definitely won't change the hair color.

Therefore, remember to include these specific features in your prompt when generating images. So for our case we will include the following in the prompts: "ginger hair, red hair, freckles on face, blue eyes".

• Positive Prompt: best quality, beautiful, 30 year old woman, (cleavage:1.2), medium close-up portrait, waist and up, ginger hair, messy hair, detailed face, (freckles:1.2), freckles on face, hyper realistic, (blue eyes:1.1), (natural lightning:1.1), perfect shadows, (facing camera), (looking into the camera), light blue shirt
• Negative Prompt: (worst quality:1.2), (low quality:1.2), (lowres:1.1), (monochrome:1.1), (greyscale), multiple views, comic, sketch, (((bad anatomy))), (((deformed))), (((disfigured))), mutation hands, mutation fingers, extra fingers, missing fingers, watermark
• Sampling Steps: 30
• Sampler: DDIM
• CFG scale: 7
• Size: 512x912
• Model: RealisticVision

As you can see, when the Roop Extension is turned on, the face closely resembles our initially chosen face portrait. To enhance this further, we can harness the power of the OpenPose model within ControlNet, which grants us greater control over the pose our AI influencer will adopt. Let's dive into ControlNet & OpenPose.

Would you like to place your AI Influencer in various poses? This is where the OpenPose model in ControlNet proves to be invaluable. To get started, simply access the "ControlNet" expansion panel and either upload or drag and drop a stick figure pose onto the canvas.

If you're looking for a great starting point, you can explore the Dynamic Poses Package. If you want to learn more about ControlNet & OpenPose you can read more here:

How to Use Open Pose & ControlNet in Stable Diffusion

ControlNet Settings

When you dropped your stick figure pose onto the canvas make sure you set the following settings:

• Check the checkboxes for "Enable" and "Pixel Perfect" (If you have <6GB of VRAM you can also check the "Low VRAM" checkbox).
• Select "OpenPose" as the Control Type
• Select "None" as the Preprocessor (This is because the image has already been processed by the OpenPose Editor).
• Select "Control_v11p_sd15_openpose" as the Model

When adding an image from a platform like Instagram/Pinterest to ControlNet, please ensure that you select "openpose_full" as the Preprocessor option.

Examples (ControlNet & OpenPose)

• Base Prompt: best quality, beautiful, 30 year old woman, ginger hair, detailed face, (freckles:1.1), (blue eyes:1.1), hyper realistic, (facing camera:1.2), (looking into the camera:1.5), natural lightning, perfect shadows, cleavage

Feel free to experiment with prompts for clothing, just as I did. Additionally, you can enhance your results by including prompts such as "smiling," which I used for the middle picture.

As you can observe, the OpenPose packages effortlessly maneuver your character into various poses. But wait, there's more! What if you fancy giving your AI Influencer a brand-new hairstyle? Let's explore how we can achieve this.

Since the Roop Extension does not swap the hairstyle. We will need a different solution to change our AI Influencer's hairstyle to our liking. You could prompt different hairstyles, but an easier way is to just make use of specific hair style LoRA's. For this purpose we will use the Hairstyled Collection Package from Civit AI. Not sure how to use LoRA's?

How to Install & Use LoRA Models in Stable Diffusion

Download Specific LoRA Hairstyles

We will try out different LoRA's from the Hairstyle Collection mentioned above. We will start off by downloading the "Space Buns" hairstyles. Once your download is complete you want to move the downloaded file inside the Lora folder, this folder can be found here; "stable-diffusion-webui\models\Lora"

LoRA Settings

When generating your image, be sure to incorporate the LoRA into your prompt. The Hairstyles Collection Package suggests using a LoRA weight between 0.2 and 0.5 for the "txt2img" model. Also add the "Trigger Words" from the downloaded LoRA to your prompt.

These trigger words will of course differ when downloading a different hairstyle

Now, let's explore some examples of different hairstyle and a different selected pose from the previously mentioned Poses Package

Space Bun Hairstyle

Trigger words added to our prompt: "space_buns_hairstyle".

Ponytail Weave Hairstyle

Trigger words added to our prompt: "ponytail_weave_hairstyle".

This tutorial has provided the groundwork for building your own AI Influencer. Initially, we covered the creation of an AI portrait, which can subsequently be employed by Roop to ensure consistent facial features in the generated images. You've also gained the skills to efficiently adjust your influencer's pose using the OpenPose model. As a bonus, you're now equipped to effortlessly modify your new Instagram influencer's hairstyle. We hope you found this tutorial informative and insightful.