Creation of AI videos with Stable Diffusion Deforum

In this guide, you will learn how to create impressive videos using Stable Diffusion Deforum. This technique allows you to generate engaging content through animations with stable diffusion. Stable Diffusion is a powerful tool that enables you to create creative videos that can go viral on platforms like TikTok and Instagram. Below, you will be guided step by step through the process of creating an animated video.

Key Insights

• Stable Diffusion Deforum is an extension for animation creation.
• You must use Google Colab to run the required notebook.
• You need storage space on Google Drive.
• The animation requires an understanding of motion and parameters like translation x, y, z, and 3D rotations.
• The video generation can be customized by adjusting keyframes and prompts.

Step-by-Step Guide

Step 1: Setting up Google Colab and the Deforum Notebook

First, you need to set up a Google Drive account if you don't have one yet. Then go to Google Colab and copy the Deforum Notebook to your Google Drive.

After copying, go to your Google Drive and look for the "colab Notebooks" folder. Here, you will find your freshly copied notebook.

Step 2: Setting up Runtime and Permissions

If you are using the notebook for the first time, click on "Runtime" and then "Run all". You will need to grant permission for the notebook to access your Google Drive.

Step 3: Configuring Animation Settings

For your first animation, you should choose the "3D Video" mode. Determine the number of frames you want – for example, 300.

Note that when using 3D, some parameters are ignored, like the angle and zoom. Instead, we focus on translation parameters and 3D rotations.

Step 4: Planning Animations

First, think about how you want the animation to look. Imagine the object is the camera. The translation x moves the camera sideways, the translation y vertically, and the translation z is for zooming.

Step 5: Configuring Motion Parameters

Here's where it gets exciting! You can animate motion parameters like translation and rotation. For example, you could animate a movement from 0 to position 1 over time.

Step 6: Using the Keyframe Generator

Utilize the keyframe string generator to plan the movements. Here you can animate the x-axis, for example, from -8 to +8. Specify the number of frames and send it to the generator.

Step 7: Animating the Motion Parameters

We are now creating an animation by inserting the parameters into their respective fields. Copy the results from the generator and paste them accordingly into the translation x, translation y, and translation z parameters.

Step 8: Adjusting Fog and Threshold Values

Set the fog strength. If the fog strength is too high, the animation will be generated faster. Here you can set values like 0.002 for a smooth transition.

Step 9: Customizing Prompts for Each Frame Action

Here, you filter what should be displayed in your video. Change the prompts per frame to create interesting transitions, like from a Ghost Rider to a Zombie.

Step 10: Create Video from Frames

Once all animations are set, go to "Create Video from Frames". Make sure you set the frames to the desired rate (e.g. 24 FPS).

Step 11: Execute and Check the Animation

Click on "Runtime" and then "Run All". The cells will be executed in the correct order, resulting in your animation being generated.

Check if the video matches your expectations.

If everything looks good, you can export and reuse the result.

Summary

In this tutorial, you have learned how to create videos using Stable Diffusion Deforum. The entire process, from setting up the Google Colab notebook to fine-tuning your animations, has been explained step by step. With the right settings and a bit of creativity, you can generate impressive videos that are both entertaining and engaging.