# Character Animation

Genie 2, developed by Google DeepMind, is a large-scale foundational world model capable of generating endless, operable, and playable 3D environments based on a single prompt image, designed for training and evaluating embodied agents. Genie 2 represents a significant advancement in the field of deep learning and artificial intelligence, showcasing various emergent capabilities in large-scale generative models, such as object interaction, complex character animation, and physical simulation, by simulating virtual worlds and their consequences. The research behind Genie 2 fosters new creative workflows for prototyping interactive experiences and opens up new possibilities for the development of future general AI systems and agents.

JoyVASA is an audio-driven character animation technique based on diffusion models that generates facial dynamics and head movements by separating dynamic facial expressions from static 3D facial representations. This technology enhances video quality and lip-sync accuracy, expands into animal facial animation, supports multiple languages, and improves training and inference efficiency. Key advantages of JoyVASA include the ability to generate longer videos, motion sequence generation independent of character identity, and high-quality animation rendering.

Animate-X is a universal animation framework based on LDM, designed for various character types (collectively termed X), including humanoid characters. This framework enhances motion representation by introducing pose indicators, allowing for a more comprehensive capture of motion patterns from driving videos. The primary advantages of Animate-X include in-depth modeling of motion and the ability to understand motion patterns in driving videos, applying them flexibly to target characters. Additionally, Animate-X introduces the Animated Anthropomorphic Benchmark (A2Bench) to evaluate its performance on universal and widely applicable animated images.

ProtoMotions is a project dedicated to creating interactive physics-based virtual agents. It supports IsaacGym and IsaacSim and is built on Hydra and OmegaConfig, facilitating simple configuration management. This project offers researchers and developers a platform to develop and test physics-based character animation techniques, applicable not only to academic research but also in fields such as gaming, film, and virtual reality.

UniAnimate is a unified video diffusion model framework for character image animation. It maps reference images, pose guidance, and noisy video to a shared feature space, reducing optimization difficulty and ensuring temporal coherence. UniAnimate can handle long sequences, supports random noise input and first-frame conditioning input, significantly improving its ability to generate long-term videos. Additionally, it explores alternative time modeling architectures based on state-space models to replace the original computationally intensive time Transformer. UniAnimate achieves superior synthetic results compared to existing state-of-the-art techniques in both quantitative and qualitative evaluations, and can generate highly consistent one-minute videos through iterative use of the first-frame conditioning strategy.

Rokoko is a sensor-based motion capture system that offers high-quality body, finger, and facial animation solutions for 3D digital creators. With its user-friendly interface and affordable price, it allows users to easily achieve realistic character animation.

Vid2DensePose is a powerful tool designed to apply the DensePose model to videos, generating detailed 'body landmark' visualizations for each frame. This tool is particularly useful for enhancing animation, especially when used in conjunction with MagicAnimate, enabling seamless and temporally coherent character animation.

Animate Anyone aims to generate character videos from static images driven by signals. Leveraging the power of diffusion models, we propose a novel framework tailored for character animation. To maintain consistency of complex appearance features present in the reference image, we design ReferenceNet to merge detailed features via spatial attention. To ensure controllability and continuity, we introduce an efficient pose guidance module to direct character movements and adopt an effective temporal modeling approach to ensure smooth cross-frame transitions between video frames. By extending the training data, our method can animate any character, achieving superior results in character animation compared to other image-to-video approaches. Moreover, we evaluate our method on benchmarks for fashion video and human dance synthesis, achieving state-of-the-art results.

Rokoko Vision is a free AI motion capture tool. Users can upload videos or use webcams for motion capture and edit and adjust them using the animation editor. The tool offers high-precision motion capture technology, allowing users to quickly create realistic character animations. Rokoko Vision's pricing varies depending on the user's location.

Story-to-Motion is a novel task that takes a story (top green area) and generates motion and trajectories consistent with the textual description. The system utilizes modern large language models as a text-driven motion scheduler, extracting a series of (text, location) pairs from long texts. It also develops a text-driven motion retrieval scheme, combining classic motion matching with motion semantics and trajectory constraints. Furthermore, it designs a progressive masking transformer to address common problems in transition motions, such as unnatural poses and sliding. The system excels in three different subtasks: trajectory following, temporal action combination, and action mixing, outperforming previous motion synthesis methods.

Puppetry is a tool that uses your facial features to add animation to images. It allows you to quickly and easily create multiple variations for game characters, storyboard characters, or intermediate images. No binding, headgear, makeup, or lengthy shooting process is required, just your camera and some magic!

Movmi is an AI-powered motion capture tool that captures human body movements from 2D media data (images and videos), providing developers with high-quality human motion capture solutions. The entire capture process is completed in the cloud, requiring no high-end equipment. Movmi supports capturing footage from various cameras, including smartphones and professional cameras, suitable for diverse life scenarios and even supporting multi-character scenes. Movmi also offers a full-text atlas character library for various animation projects. Movmi's membership plans are divided into Bronze, Silver, and Gold, offering different levels of features and experiences. Users can use the output FBX files in any 3D environment.