

Instructvideo
Overview :
InstructVideo is a method for training text-to-video diffusion models using reward fine-tuning guided by human feedback. It employs an editing-based reward fine-tuning approach, which reduces fine-tuning cost while enhancing efficiency. Leveraging pre-established image reward models, it provides reward signals through segment-wise sparse sampling and temporal decay rewards, significantly improving the visual quality of generated videos. InstructVideo not only enhances the visual quality of generated videos but also maintains strong generalization capabilities. For more information, please visit the official website.
Target Users :
Training and optimization of text-to-video generation models.
Features
Reward fine-tuning with human feedback
Editing-based reward fine-tuning
Utilizing image reward models to provide reward signals
Segment-wise sparse sampling and temporal decay rewards
Featured AI Tools

Sora
AI video generation
17.0M

Animate Anyone
Animate Anyone aims to generate character videos from static images driven by signals. Leveraging the power of diffusion models, we propose a novel framework tailored for character animation. To maintain consistency of complex appearance features present in the reference image, we design ReferenceNet to merge detailed features via spatial attention. To ensure controllability and continuity, we introduce an efficient pose guidance module to direct character movements and adopt an effective temporal modeling approach to ensure smooth cross-frame transitions between video frames. By extending the training data, our method can animate any character, achieving superior results in character animation compared to other image-to-video approaches. Moreover, we evaluate our method on benchmarks for fashion video and human dance synthesis, achieving state-of-the-art results.
AI video generation
11.4M