AI Video Editing via Text-to-Image Diffusion

This AI utilizes the power of a text-to-image diffusion model to perform text-driven video editing. Given a source video and a target text-prompt, the AI generates a high-quality video that aligns with the target text while preserving the spatial layout and dynamics of the input video. The AI operates based on the principle that consistency in the edited video can be obtained by enforcing consistency in the diffusion feature space. It achieves this by explicitly propagating diffusion features based on inter-frame correspondences, which are readily available in the model. The framework does not require any training or fine-tuning, and can work in conjunction with any off-the-shelf text-to-image editing method.