Prompt Travelling: Create GIFs with AnimateDiff (A1111)

Welcome to the realm of AnimateDiff and A1111, where we'll guide you in creating animated GIFs through the clever technique of Prompt Traveling. This journey is all about the harmonious blend of creativity and technology, where you'll learn to breathe life into your digital art. So, join us on this adventure, where the worlds of art and innovation converge to unlock the magic of GIF creation! Lets get started!

Before embarking on the exciting journey of creating GIF animations with the magic of Stable Diffusion, Prompt Travelling, and AnimateDiff, let's make sure we've got the essential extension in order. Below, you'll find our checklist of prerequisites.

Requirement 1: AnimateDiff Extension

• Navigate to the "Extensions" tab within Stable Diffusion.
• Click on the subtab "Available".
• Click on "Load from" button.
• Search for "AnimateDiff" and Click on "Install".

To make the most of the AnimateDiff Extension, you should obtain a Motion module by downloading it from the Hugging Face website.

• Go to the official Hugging Face website and locate the AnimateDiff Motion files.
• Download the "mm_sd_v14.ckpt", "mm_sd_v15.ckpt" or the "mm_sd_v15_v2.ckpt" file
• After acquiring the Motion Module, move the file to the following directory structure: "stable-diffusion-webui" > "extension" > "sd-web-ui-animatediff" > "model"

After you've completed the installation of the extension and motion module, go to the "Installed" tab, and click on "Apply and restart UI." Furthermore, I strongly advise you to conduct a full restart of your Stable Diffusion for a smooth experience and to mitigate any potential errors.

Prompt traveling is a technique that allows you to fine-tune your positive prompt to have precise control over specific details at designated time frames (frame numbers) within the final GIF or Video creation. In combination with AnimateDiff, we can now apply more influence over the visual aesthetics of our GIFs or videos, including factors like style, background, and clothing choices and many others elements.

Prompt travelling is a technique that can be employed within the positive prompt input field, as illustrated in the image below. You'll notice that our prompt differs from the typical format because it incorporates prompt travelling.

• Green Line: The first line represents the head prompt, which is not required. You have the flexibility to include multiple lines of head prompts. The head or base prompt serves as the foundational input for determining the overall appearance of the resulting video or GIF.
• White Lines: The white lines are reserved for prompt travelling, following the pattern "frame number: prompt" Ensure that your frame numbers are in sequential order. These prompts show the modifications that will happen at certain time frames, along with the main prompt if you've added one.

If the installation is completed successfully, you will find an additional dropdown menu in both the txt2img and img2img tabs. To access additional information about the AnimateDiff extension, please feel free to explore the official AnimateDiff GitHub page.

let's navigate to the "txt2img" tab and scroll down to locate the 'AnimateDiff' dropdown, where we can adjust the settings for crafting our GIF animation. Below, we have provided the settings for generating a GIF animation:

• Check the checkbox for "Enable AnimateDiff"
• Motion module: mm_sd_v15_v2.ckpt (Feel free to experiment with different motion modules)
• Number of frames: 48.
• Frames Per Second (FPS): 8 (Resulting in a 6-second GIF animation)
• Closed loop: A (means that the extension will aggressively try to make the last frame the same as the first frame. The prompt travel will be interpolated to be a closed loop.)
• Save format: GIF (Optionally, you can also choose "MP4")
• The remaining settings can stay in their default state.

Our next step is to fine-tune the txt2img settings, specifically by utilizing "prompt travelling" within the prompt text field to change the end result of our GIF animation. We aim to make prompt adjustments every 16 frames, which equates to every 2 seconds. So, let's proceed!

As we get more into the creative process, it's essential to understand the details of the txt2img settings. These settings are crucial for shaping the GIF animation, especially when used with prompt travelling.

Txt2Img Settings

In the "txt2img" section of the Stable Diffusion interface, we will move on to select the specified settings below:

• Checkpoint: ToonYou
• Positive Prompt: masterpiece, 30 year old women, cleavage, red hair, bun, ponytail, medium breast, desert, cactus vibe, sensual pose, (looking in the camera:1.2), (front view:1.2), facing the camera,close up, upper body 0: red dress 16: white dress 32: green dress
• Negative Prompt: (worst quality:1.2), (low quality:1.2), (lowres:1.1), (monochrome:1.1), (greyscale), multiple views, comic, sketch, (((bad anatomy))), (((deformed))), (((disfigured))), watermark, multiple_views, mutation hands, mutation fingers, extra fingers, missing fingers, watermark, nude, nsfw
• Sampling Method: DPM++ 2M Karras
• Sampling Steps: 30
• Width & Height: In our case this is an aspect ratio of 9:16
• CFG Scale: 7
• Seed: -1

Hires Fix Settings

According to the details provided on the Civit.ai website in the ToonYou checkpoint section, the most favorable outcome is achieved by utilizing the "hires fix" option. The recommended settings are outlined below:

• Upscaler: R-ESRGAN 4x+ Anime6B
• Hires steps: 14
• Denoising strength: 0.35
• Upscale by: 1.5+

Once all the configurations are in order, simply click the "Generate" button, then relax and anticipate your GIF animation with the applied prompt travelling changes.

After completing the rendering process, we've generated a GIF animation through prompt travel, in which the color of the dress changes every 2 seconds (16 frames). We hope you enjoy watching this GIF animation.

In conclusion, we've harnessed the power of Stable Diffusion, Prompt Travelling, and AnimateDiff to craft captivating GIF animations. By following the steps, we've embraced prompt adjustments to achieve precise control over our creations.

Seeking to enhance the caliber of your GIFs and maintain facial consistency in your animations? If that's your aim, you're in the right place. Armed with the knowledge you've gained from this article and the art of Prompt Traveling, you're now equipped to embark on your own creative journey and produce remarkable GIF animations. For further insights on achieving facial consistency in your final GIF animations, explore the article provided below:

Create Realistic GIFs with Stable Diffusion (AnimateDiff & Roop)