Fast Video Generation with AnimateDiff & LCM LoRA's (A1111)

Enter the swift domain of video creation, guided by AnimateDiff and LCM LoRA's (A1111). This powerful combination ensures speedy and captivating animations, reshaping the way you approach creativity. Wave goodbye to slow workflows as we explore efficiency and innovation, ensuring accelerated video production that effortlessly enhances your content.

Before we can speed up the process of creating videos or GIFs using LCM LoRA, there are a few things we need to have in place. Check out the list below to make sure we meet the necessary requirements.

Requirement 1: AnimateDiff Extension

• Go to the "Extensions" tab located within Stable Diffusion.
• Select the "Available" subtab.
• Click the "Load from" button.
• Look for "AnimateDiff" and proceed to click on the "Install" option.

To maximize the benefits of the AnimateDiff Extension, acquire a Motion module by downloading it from the Hugging Face website. Visit the official Hugging Face website and find the AnimateDiff Motion files.

• Download either the "mm_sd_v14.ckpt", "mm_sd_v15.ckpt", "mm_sd_v15_v2.ckpt" ore the latest version 3 file labeled as: "v3_sd15_mm.ckpt".
• Once you have the Motion Module, relocate the file to the designated directory structure: "stable-diffusion-webui" > "extension" > "sd-web-ui-animatediff" > "model."

Once the extension and motion module installation is finished, navigate to the "Installed" tab, and select "Apply and restart UI." Additionally, I highly recommend performing a complete restart of your Stable Diffusion for a seamless experience and to address any potential errors.

Requirement 2: Latent Consistency Model (LCM) LoRA

For faster video generation in Stable Diffusion, a single LCM LoRA is sufficient and can be integrated directly into the prompt, similar to regular LoRAs. Explore our comprehensive guide on harnessing the power of LCM LoRA's for accelerated image generation below:

How to Speed Up Image Generation with LCM LoRA's (A1111)

Downloading LCM LoRA

Depending on whether you are using SDXL or SD 1.5, various LCM LoRAs are available for download. Below, we will provide a list of where you can obtain the "pytorch_lora_weight.safetensors" file from either of these pages.

• SDXL LCM LoRa: LCM SDXL LoRA
• SD 1.5 LCM LoRa: LCM SD 1.5 LoRa

LCM LoRa File Names & Location

Just like with standard LoRA's, move the downloaded file to the LoRa models folder, situated at: "stable-diffusion-webui\models\Lora"

I recommend customizing the name of the LCM LoRA to align with your version. For instance, following the download of the SD 1.5 LCM LoRA mentioned earlier, I modified the file name to "LCM_SD1.5.safetensors." Similarly, for the SDXL LCM LoRA, I opted for "LCM_SDXL.safetensors.

Once all the installations are configured, we can move forward with configuring the AnimateDiff settings. We'll utilize AnimateDiff to craft captivating animations, available in either GIF or MP4 format. Let's launch the Stable Diffusion UI and go to the "txt2img" tab. Scroll down to find the AnimateDiff dropdown menu, where we'll proceed to configure the settings as follows:

• Motion module: mm_sd_v15_v2.ckpt (The latest Motion Module avaiable at the moment)
• Save format: MP4 (Moreover, you have the option to save it as a GIF as well)
• "Enable" AnimateDiff
• Number of frames: 16 (The SD1.5 motion modules are trained using 16 frames, ensuring optimal results when the number of frames is set to 16)
• FPS (Frame Per Second): 8

The remaining settings can remain in their default state. However, if you desire more information about the AnimateDiff extension, you can visit the AnimateDiff GitHub page.

If all the configurations are properly set within the AnimateDiff Menu, we can proceed by scrolling to the top. Initially, we will select our checkpoint. In our scenario, as we aim for a realistic video animation, we will choose the "Realistic Vision" checkpoint.

Txt2img Prompt Settings (LCM LoRa)

Subsequently, we will incorporate the downloaded LCM LoRa file. In our instance, we are using SD 1.5 and will include our LCM LoRa within the positive prompt.

• Navigate to the Lora subtab within the "txt2img" tab.
• Select the correct LCM LoRa from the available Lora options.
• Provide a positive prompt, incorporating the included LCM LoRA, and provide a detailed description of your desired final video animation.
• Create a negative prompt

Txt2Img Settings

Next, we'll progress to the global settings within the "txt2img" tab, playing a pivotal role in improving rendering speed when employing with the LCM LoRA, Here are the settings to enhance the speed of your video generation:

• Sampling Method: LCM (If you cant select the LCM sampling method, go to extensions tab, then click on "Check for Updates" and Apply and restart UI")
• Sampling Steps: 8 (Set between 2-8, Sampling Steps of 8 yields the highest quality results).
• CFG Scale: 2 (Choose between 1-2, CFG Scale of 2 will include the negative prompt and CFG Scale of 1 will exclude the negative prompt).
• Width & Height: Select your preferred ratio; in our instance, we'll generate a video with a 9:16 ratio.

Having set all the precise configurations, proceed by scrolling to the top and click on the crucial "Generate" button. This action will enable you to craft remarkable videos or GIFs, depending on the settings you specified in the AnimateDiff option menu.

Upon finishing the rendering process, we've produced a video animation with accelerated rendering speed, courtesy of the LCM LoRA and AnimateDiff. We trust you'll enjoy watching this video animation. The generated video can be located in the designated file path: "C:\stable-diffusion\stable-diffusion-webui\outputs\txt2img-images\AnimateDiff"

In the realm of video animation, the synergy of AnimateDiff and LCM LoRa's (A1111) emerges as a game-changer, offering not just speed but an innovative edge to your creative endeavors. This powerful combination, when harnessed effectively, transforms the conventional approach to content production. Wave goodbye to sluggish workflows and say hello to efficiency, as AnimateDiff and LCM LoRA's pave the way for accelerated video creation that seamlessly elevates the quality and impact of your visual content. So, take the leap, explore the possibilities, and let your animations captivate with the prowess of AnimateDiff and LCM LoRA's.