Wan2.2Open-source video generation model family (5B/14B) for text/image/video-to-video, delivering 1080p clips with improved motion and control.

Wan2.2 is an open-source video generation model family with two variants, 5B and 14B, supporting text/image/video-to-video generation. It can deliver 1080p clips with improved motion and control. The model incorporates a Mixture-of-Experts (MoE) architecture, cinematic-level aesthetics, and complex motion generation. The architecture of Wan2.2 is based on a video diffusion model, with the MoE architecture separating the denoising process across timesteps. This allows it to maintain the same computational cost while increasing the overall model capacity. Wan2.2 has been trained on a significantly larger dataset than its predecessor, Wan2.1, with +65.6% more images and +83.2% more videos. This expansion enhances the model's generalization across multiple dimensions, including motions, semantics, and aesthetics. The model supports text-to-video and image-to-video generation at 720P resolution with 24fps, and can run on consumer-grade graphics cards like 4090. It is one of the fastest 720P@24fps models currently available, capable of serving both the industrial and academic sectors simultaneously. Wan2.2 has been integrated into several frameworks, including Diffusers, ComfyUI, and ModelScope, and has been used for various applications, such as character animation, replacement, and audio-driven cinematic video generation. Wan2.2 has also been used to create a unified model for character animation and replacement with holistic movement and expression replication, and an audio-driven cinematic video generation model, including inference code, model weights, and technical report. The model has also been used to create a 5B model built with the Wan2.2-VAE that achieves a compression ratio of 16 16 4, supporting both text-to-video and image-to-video generation at 720P resolution with 24fps. The model has been released under an open-source license and has gained popularity, with over 50 commits on its GitHub repository and a large community of contributors and users. Wan2.2 is a powerful video generation model that provides state-of-the-art performance and versatility for various applications, including character animation, replacement, and audio-driven cinematic video generation. Its architecture and training dataset make it a valuable resource for researchers and developers working in the field of video generation and processing., The model is suitable for various applications, including character animation, replacement, and audio-driven cinematic video generation. However, like any machine learning model, Wan2.2 has some limitations. For example, it requires a significant amount of computational resources and training data to achieve its performance, and it may not be suitable for all types of video generation tasks. Additionally, while Wan2.2 has been integrated into several frameworks, its performance and versatility may vary depending on the specific use case and application. Overall, Wan2.2 is a powerful and versatile video generation model that provides state-of-the-art performance for various applications. Its architecture and training dataset make it a valuable resource for researchers and developers working in the field of video generation and processing.