# NeRF

SyncAnimation is an innovative audio-driven technology capable of real-time generation of highly realistic speaking avatars and upper body movements. By combining audio with synchronized pose and expression techniques, it addresses the shortcomings of traditional methods in terms of real-time performance and detail representation. This technology primarily targets application scenarios that require high-quality real-time animation generation, such as virtual streaming, online education, remote conferencing, and holds significant practical value. Its pricing and specific market positioning have yet to be determined.

MimicTalk is a personalized 3D talking face generation technology based on Neural Radiance Fields (NeRF). It can mimic the static appearance and dynamic speaking style of a specific identity within minutes. The main advantages of this technology include high efficiency, high-quality video generation, and precise imitation of the target character's speaking style. MimicTalk uses a generic 3D facial generation model as its foundation and employs a static-dynamic hybrid adaptation process to learn the personalized static appearance and facial dynamics. Additionally, it introduces a Contextual Stylization Audio-to-Motion (ICS-A2M) model to generate facial movements that align with the target character's speaking style. The technological background of MimicTalk is based on the latest advancements in deep learning and computer vision, particularly in the areas of facial synthesis and animation generation. Currently, this technology is freely available to the research and development community.

Shusheng: Skyline LandMark is a real-world 3D model based on NeRF technology, achieving 4K HD training over an extensive area of 100 square kilometers, with capabilities for real-time rendering and free editing. This technology represents a new pinnacle in city-scale 3D modeling and rendering, boasting high efficiency for training and rendering, making it a robust tool for urban planning, architectural design, and virtual reality.

IllumiNeRF is a 3D relighting technology that recovers a 3D representation from a series of images of an object captured under unknown lighting conditions, allowing it to be rendered from new viewpoints under target illumination. This technique avoids traditional methods based on inverse rendering, which often involve optimization through differentiable Monte Carlo rendering, a process that is both fragile and computationally expensive. IllumiNeRF employs a simpler approach: first, it uses an image diffusion model to relight each input image, and then it uses these re-lit images to reconstruct a Neural Radiance Field (NeRF) model, enabling rendering of new views under target illumination. This approach has achieved surprisingly competitive performance and state-of-the-art results in multiple relighting benchmark tests.

Nerfstudio is an open-source neural radiance field (NeRF) development framework. It provides a simple and easy-to-use API, supporting modular NeRF construction and training. Nerfstudio helps users more easily understand and explore NeRF technology and provides tutorials, documentation, and more learning resources. Welcome users to contribute new NeRF models and datasets. The main features of Nerfstudio include model training, data processing, and visualization.

SIGNeRF is a novel method for fast and controllable NeRF scene editing and object generation into existing scenes. It introduces a new generation update strategy that ensures 3D consistency when editing images without iterative optimization. SIGNeRF leverages the advantages of ControlNet's depth-conditional image diffusion models, allowing for editing existing NeRF scenes in a single forward pass through a few simple steps. It can generate new objects into existing NeRF scenes or edit existing objects, enabling precise control over the scene.

DL3DV-10K is a large-scale real-world dataset containing over 10,000 high-quality videos. Each video is manually annotated with key scene points and complexity, and also provides camera pose, NeRF depth estimation, point clouds, and 3D meshes. The dataset can be used for general NeRF research, scene consistency tracking, visual language models, and other computer vision studies.

ZeroNVS is a tool for synthesizing zero-shot 360-degree panoramas from a single real image. It provides 3D SDS distillation code, evaluation code, and a pre-trained model. Users can utilize this tool for their own NeRF model distillation and evaluation, and experiment on various datasets. ZeroNVS boasts high-quality synthesis effects and supports customized image data. The tool is primarily used in virtual reality, augmented reality, and panoramic video production.

Instruct-NeRF2NeRF is an instruction-based editor for editing NeRF scenes. It uses an image-conditional diffusion model (InstructPix2Pix) to gradually edit the input image while optimizing the underlying scene, resulting in an optimized 3D scene that conforms to the editing instructions. We demonstrate that our method can edit large-scale real-world scenes and achieve more realistic and targeted edits than previous work.