Embark on an animation journey with DiffEx, a desktop UI for AnimateDiff.
This tool opens possibilities, allowing effortless video and image imports for your ControlNet map. Enjoy multi ControlNet and seamless IP Adapter support. DiffEx automates the import of Models, Motion Modules, VAEs, LoRAs, and Embeddings from your Stable Diffusion WebUI, eliminating manual setup hassles. Explore enhanced capabilities with SDXL and LCM support.
Precision reigns with LoRA mapping, isolating LoRAs to regions and modulating strength over animations. Elevate with the Refiner for upscaled animations. Introducing Stylize mode (vid2vid), DiffEx transforms videos into prompts, pushing the boundaries of animation, and automates the ControlNet setup.
For DiffEx to run smoothly on your computer we first need to install some dependencies. If you already have Stable Diffusion installed you can skip the python installation.
Python: Before setting up DiffEx, make sure you have Python installed and added to PATH (check the box during installation). Download python from the official python.org website, the recommend version is 3.10.11.
FFmpeg: Follow our detailed guide on how to install FFmpeg and add it to PATH.
Go to the official DiffEx CivitAI page and save the file anywhere on your PC.
Once the zip file has been downloaded:
To avoid permission issues, run DiffEx in Administrator mode: Right-click DiffEx.exe -> Run as administrator -> Yes.
When you first launch DiffEx, you need to connect it to your AnimateDiff CLI Prompt Travel folder. Click the "Link" button on the top right of the Setup page.
Then navigate to your AnimateDiff root folder (for example, S:\DiffEx\animatediff-cli-prompt-travel). Here you can also link your Stable Diffusion models, VAE’s & LoRA’s. When everything is linked, press “Apply”. If the linking is successful, the box next to the "Link" button will be checked, and the AnimateDiff console will start up.
If for some reason you receive an error here, make sure that you are running DiffEx in administrator mode.
Now the last thing we need to install is the motion module for AnimateDiff.
Go to the Huggingface AnimateDiff page and download the motion module. Right click the download button and click on “Save Link As”.
Now navigate to your motion-module folder and press save. animatediff-cli-prompt-travel\data\models\motion-module
Now you’re all set to run DiffEx, I recommend always running it as administrator, because when enabling new features it will sometimes require a download of a specific model for example IP-Adapter or Refiner. Running DiffEx in Administrator mode makes sure it has the permissions to download these models.
Now let’s create a video with DiffEx.
In the main menu, name your project.
Click the hamburger menu in the top left and navigate to the prompt tab. Here is where you want to fill in your prompt.
I recommend going to the CivitAI page of your model and look for a good prompt and adjust this to your liking, this results in getting the best output. You also have the option to add an image prompt, which I will cover in an other tutorial.
Click the hamburger menu again and go to the execute tab. Here you want to set the dimensions of your video and the length. Currently there is no build-in aspect ratio lock so you have to do some math yourself to achieve your desired aspect ratio.
Now click on “Build & Run” to start your animation!
Now let's check out some of the examples using this method! You can find your animations in the output folder. animatediff-cli-prompt-travel\output
DiffEx is a user-friendly Desktop UI designed for AnimateDiff, streamlining animation creation. It automates tasks like importing videos and images, providing multi ControlNet and IP Adapter support, and simplifying the integration of models, motion modules, and more.
Follow these steps:
Feel free to adjust settings to your liking throughout the process.