Transform Videos into AI Animations with Stable Diffusion (mov2mov)

Video-to-video tasks can often be time-consuming and require manual effort. However, with the Mov2mov extension, you can automate many of these steps and simplify the process. This blog post will guide you trough the Mov2mov extension to create impressive videos effortlessly. Whether you're a beginner or an experienced video enthusiast, this tutorial will help you achieve remarkable results.

The first step in using the Mov2mov extension is to install it. Within the AUTOMATIC1111 Web-UI:

• Navigate to the Extension page and select the "Install from URL" tab.
• Enter the following URL for the extension's git repository: https://github.com/Scholar01/sd-webui-mov2mov
• Click "Install" and wait a few seconds until the extension is finished installing.
• Now go to the "Installed" tab, click on check for updates and then on "Apply and restart UI"
• Once Installed, completely close and restart the Web-UI to ensure the extension is properly installed.

After installing the extension, you'll find a new page called "mov2mov." On this page, select a Stable Diffusion checkpoint from the dropdown menu.

For example, you can choose the "revAnimated" model. Checkout this tutorial on how to install & use checkpoints/models:

How to Install Stable Diffusion Checkpoints & Models

Step1 - Enter a Prompt

Next, enter the desired prompt and negative prompt for your video. You can use a detailed description or specific keywords to guide the video generation process.

Step2 - Upload Video

Upload the video you wish to work with by dropping it onto the video canvas. Set the Resize mode to: "Crop and resize".

Step3 - Mov2mov Settings

When using the Mov2mov extension, here are some key settings to consider:

• Sampling method: Keep in mind that deterministic samplers like Euler, LMS, and DPM++2M Karras might not work well with this extension, as they may not effectively reduce flickering.
• Noise Multiplier: Utilize the slider to adjust the noise multiplier. For smoother results and reduced flickering, keep it at 0.
• CFG Scale:: Control the extent to which the prompt is followed by adjusting the CFG scale. In the provided video, a scale of 7 was used.
• Denoising Strength: Fine-tune the amount of change applied to the video by adjusting the denoising strength. A value of 0.6 was used in the example video.
• Movie Frames: The frames per second of your output. The higher this value, the smoother your video, but this will have to render more images.
• Max Frame: Determine the total number of frames to be generated. For initial testing, set it to a low number such as 10. To generate a full-length video, set it to -1.
• Seed: The seed determines the value for the first frame. All frames will use the same seed value, even if you set it to -1 for a random seed. By understanding and adjusting these settings, you can have greater control over the output and achieve desired results when utilizing the Mov2mov extension.

If you haven't installed ControlNet yet, you can refer to our ControlNet installation guide.

"Enable" ControlNet and set the following options:

• Pixel Perfect: Yes
• Control Type: Lineart
• Preprocessor: Lineart_realistic
• Model: control_xxxx_lineart
• Control weight: (0.6).

It's important to note that Mov2mov will use the current frame as the reference image, so you don't need to upload a separate reference image.

With all the settings in place, it's time to generate the video. Click the "Generate" button to start the process. Be patient as it may take some time. Once the generation is complete, your new video will appear on the right side of the page.

Click "Save" to download and save the video to your device. If you can't locate the video, check the output/mov2mov-videos folder.

Example

In case of any errors during video generation, try different Video Modes to see if that resolves the issue. If the generation fails altogether, you can manually create a video from the image series found in the output/mov2mov-images folder.

In conclusion, the Mov2mov extension in Stable Diffusion is a game changer for video-to-video transformations. With its user-friendly interface and powerful capabilities, it enables effortless and stunning video creation. Embrace this innovative tool and take your video editing skills to the next level.