Create creative 2D videos with Stable Diffusion

In this tutorial, you will learn how to create your own creative 2D videos using Deforum Stable Diffusion, similar to what you have seen on platforms like TikTok. We will guide you through the entire process step by step, from preparing your video to applying special effects using image synthesis technology. So let's get started!

Key Points:

• You can upload and customize videos from your computer or online resources.
• The user interface in Google Colab is ideal for this purpose.
• You can determine how your videos will be altered using prompts.
• It is important to set the frames per second and other parameters correctly to achieve good results.

Step-by-Step Guide

To create your video using Deforum Stable Diffusion, follow these steps:

First, open Google Colab and ensure everything is set up correctly. This is where you will run your video input and editing commands.

Then click on the "Video Input" option in the section for animations.

The special parameters you see here are not relevant for the initial steps as they will be ignored in this case. Focus on setting the correct path to your video.

Next, locate your desired video. You can upload one of your own videos or, if you do not have a suitable video, visit a platform like Pexels. Download a short video from there and save it to your computer.

Then switch to your Google Drive. Create a new folder for your videos to keep everything organized. Drag the video you just downloaded into this folder.

Once the video has been successfully uploaded, go back to Google Colab. If you haven't mounted your Drive yet, click "Play" to establish the connection.

If you do not immediately get access to your Drive, click "Refresh," and you should see your video folder.

Right-click on the video folder and copy the path.

Now paste this path into the input field for video input by deleting the predefined path and inserting your own.

Here you can also specify how many frames you want to extract from the video. If you want to extract every frame, simply enter "1." If you want fewer frames, set the number to "2" or "3," depending on your needs.

Leave the "Override extracted Frames" option enabled so that the prompts will also be applied to the extracted frames.

Now set the maximum number of frames to be processed. You can set this to 300, which is usually sufficient for most videos.

Then adjust the "Strength" setting. This determines how much the frames will be altered. Set it to around "0.65," which often leads to good results.

Now it gets exciting: Go to the prompts. Delete the first prompt and enter your own prompt, for example "Cyberpunk City vibrant color high detail 8k". By putting a hash symbol (#) before the prompt, it will be ignored by others, making the input easier.

You can also specify the image size. For most social media platforms like Instagram, a size of 320x512 pixels is recommended.

You can leave most of the remaining settings as they are, except for some adjustments to the batch settings that should be saved in stable diffusion.

You can use a fixed seed to achieve consistent results, or experiment with different settings to see what works.

It is important to set the initiate settings to a moderate strength, for example "0.45", to find a balanced ratio between original and editing level.

Before starting the processing, make sure the "Use init" option is deactivated, as you are working with the video input.

Now everything is prepared! Execute each cell one by one, or click on "Runtime" to run all at once.

The video is now being created at 12 frames per second. If the result does not match your video, adjust the frames per second accordingly to achieve the best results.

If errors occur, don't worry. It is common to encounter minor technical difficulties. In this case, simply open a new file and edit the settings again.

Once the video is created, you can evaluate the quality and see if the desired results have been achieved. You may want to adjust the frames per second or experiment with the settings until everything is perfect.

Summary

You have now learned how to use Deforum Stable Diffusion to create creative 2D videos. From selecting your video to adjusting the prompts and fine-tuning the settings, you can control all aspects of the editing process.