CogVideoX is an open-source video generation model that shares lineage with commercial models, enabling the generation of video content through textual descriptions. It represents the latest advancements in text-to-video generation technology, capable of producing high-quality videos applicable in various fields including entertainment, education, and commercial promotion.

I2VEdit is an innovative video editing technology that extends the editing of a single frame to the entire video through a pre-trained image-to-video model. This technology can adaptively maintain the visual and motion integrity of the source video and effectively handle global editing, local editing, and moderate shape changes, which existing methods cannot achieve. The core of I2VEdit includes two main processes: rough motion extraction and appearance refinement, which are precisely adjusted through coarse-grained attention matching. In addition, a skip interval strategy is introduced to mitigate the quality degradation in the automatic regression generation process of multiple video segments. Experimental results show that I2VEdit's superior performance in fine-grained video editing demonstrates its ability to produce high-quality, temporally consistent output.

FunClip is a fully open-source, locally deployed automated video editing tool. It utilizes the FunASR Paraformer series of open-source models from Alibaba's TGETHER Lab for video voice recognition. Users can then freely select text segments or speakers from the recognized results, and clicking the crop button retrieves the corresponding video clip. FunClip integrates Alibaba's open-source industrial-grade Paraformer-Large model, one of the best-performing open-source Chinese ASR models currently available, and accurately predicts timestamps in an integrated manner.

The Video Mamba Suite is a new state-space model suite for video understanding designed to explore and assess the potential of Mamba in video modeling. It contains 14 models/modules, covering 12 video understanding tasks, demonstrating efficient performance and superiority in both video and video-language tasks.

Open-Sora-Plan is an open-source project dedicated to replicating OpenAI's Sora (T2V model) and constructing knowledge about Video-VQVAE (VideoGPT) + DiT. Initiated by the Peking University-Tuizhan AIGC Joint Laboratory, the project currently has limited resources and seeks contributions from the open-source community. The project provides training code and welcomes Pull Requests.

FlowVid is an optical flow guided video synthesis model. By utilizing the spatial and temporal information of optical flow, it achieves temporal consistency between video frames. It seamlessly integrates with existing image synthesis models to enable various modification operations, including stylization, object swapping, and local editing. FlowVid boasts fast generation speed; a 4-second, 30FPS, 512×512 resolution video can be generated in just 1.5 minutes, outperforming CoDeF (3.1x), Rerender (7.2x), and TokenFlow (10.5x) respectively. In user evaluations, FlowVid achieved a quality score of 45.7%, significantly surpassing CoDeF (3.5%), Rerender (10.2%), and TokenFlow (40.4%).

Traditional 3D content creation tools empower users with direct control over scene geometry, appearance, actions, and camera paths, transforming their imagination into reality. However, creating computer-generated videos remains a tedious manual process, which can be automated through emerging text-to-video diffusion models. Despite promising prospects, the lack of control in video diffusion models limits users' ability to apply their own creativity, rather than expanding it. To address this challenge, we propose a novel approach that combines the controllability of dynamic 3D meshes with the expressiveness and editability of emerging diffusion models. Our method takes animated low-fidelity rendering meshes as input and injectsground truth correspondences derived from the dynamic mesh into various stages of a pre-trained text-to-image generation model, resulting in high-quality, temporally consistent frames. We demonstrate our method on various examples, where actions are achieved through animating bound assets or altering camera paths.

SparseCtrl is developed to enhance the controllability of text-to-video generation. It enables flexible structural control by combining sparse signals with only one or a few inputs. It includes an additional conditional encoder to process these sparse signals without affecting the pre-trained text-to-video model. This method is compatible with various formats, including sketches, depth, and RGB images, providing more practical control for video generation and pushing applications like storyboarding, depth rendering, keyframe animation, and interpolation. Extensive experiments demonstrate SparseCtrl's generalization ability on both original and personalized text-to-video generators.

The Sieve Eye Contact Correction API is a fast and high-quality solution designed for developers, enabling eye contact correction in videos. This technology redirects eye gaze, allowing video subjects to simulate eye contact with the camera even when not directly facing it. It supports various customization options to fine-tune eye gaze redirection while preserving natural blinking and head movements, and introduces a random 'look away' feature to prevent a fixed gaze. Additionally, it offers split-screen views and visualization options for easy debugging and analysis. This API is primarily aimed at video creators, online education providers, and anyone looking to enhance video communication quality. The pricing is $0.10 per minute of video.

CogVideoX is an open-source video generation model that shares lineage with commercial models, enabling the generation of video content through textual descriptions. It represents the latest advancements in text-to-video generation technology, capable of producing high-quality videos applicable in various fields including entertainment, education, and commercial promotion.

Stable Video 4D is the latest AI model launched by Stability AI, capable of transforming single object videos into multiple novel view videos from eight different angles. This technology represents a significant leap from image-based video generation to complete 3D dynamic video synthesis. It holds potential applications in game development, video editing, and virtual reality, and is continuously being optimized.

RunwayML is a leading next-generation creative suite that offers a rich set of tools enabling users to transform any idea into reality. With its unique text-to-video generation technology, users can create videos from text descriptions directly on their mobile devices. Key advantages include: 1. Text-to-Video Generation: Users can generate videos simply by inputting text descriptions. 2. Real-Time Updates: Regularly released new features and updates ensure users always have access to the latest AI video and image tools. 3. Seamless Asset Transfer: Users can effortlessly move assets between mobile devices and computers. 4. Multiple Subscription Options: Offering standard, professional, and plans that include 1000 video generations per month.

Developed by Kuaishou Technology, Kling AI is a text-to-video generation model that can produce highly realistic videos based on text prompts. It boasts efficient video generation capabilities, generating up to 2 minutes of 30 frames-per-second video, along with advanced technologies like a 3D Spatiotemporal Joint Attention mechanism and physics-world simulation, giving it a significant competitive edge in the AI video generation field.

Developed by Kuaishou, KuaiYing is a video editing application that offers a comprehensive suite of video editing features, including cutting, audio, subtitles, special effects, and more. It aims to help users easily create engaging and professional video content. It features an AI-powered anime video function that can transform videos into anime styles, offering various options like anime style, national style, and Japanese anime style. Additionally, KuaiYing boasts AI creation tools such as AI drawing, AI text-to-image, and an AI copywriting library to assist users in their creative endeavors. KuaiYing also provides a creative center to help users view data, find inspiration, and offers a powerful resource library including stickers and trending content to enhance user engagement.

Underlord is an AI-driven video and podcast editing assistant within the Descript platform. It simplifies the video and audio content editing process through a series of intelligent editing tools, boosting both productivity and quality. Underlord is committed to making creative work smoother with AI technology while minimizing the tediousness of post-production editing.

I2VEdit is an innovative video editing technology that extends the editing of a single frame to the entire video through a pre-trained image-to-video model. This technology can adaptively maintain the visual and motion integrity of the source video and effectively handle global editing, local editing, and moderate shape changes, which existing methods cannot achieve. The core of I2VEdit includes two main processes: rough motion extraction and appearance refinement, which are precisely adjusted through coarse-grained attention matching. In addition, a skip interval strategy is introduced to mitigate the quality degradation in the automatic regression generation process of multiple video segments. Experimental results show that I2VEdit's superior performance in fine-grained video editing demonstrates its ability to produce high-quality, temporally consistent output.

FunClip is a fully open-source, locally deployed automated video editing tool. It utilizes the FunASR Paraformer series of open-source models from Alibaba's TGETHER Lab for video voice recognition. Users can then freely select text segments or speakers from the recognized results, and clicking the crop button retrieves the corresponding video clip. FunClip integrates Alibaba's open-source industrial-grade Paraformer-Large model, one of the best-performing open-source Chinese ASR models currently available, and accurately predicts timestamps in an integrated manner.