# Realistic Rendering

PhysGen is an innovative method for image-to-video generation that transforms a single image and input conditions (such as force and torque applied to objects in the image) into realistic, physically plausible, and temporally coherent videos. This technology achieves dynamic simulation in image space by combining model-based physical simulation with data-driven video generation processes. The main advantages of PhysGen include producing videos that are both physically and visually realistic, and offering precise control, demonstrating its superiority over existing data-driven image-to-video generation methods through quantitative comparisons and comprehensive user studies.

MaPa is an innovative method for generating materials for 3D meshes based on textual descriptions. This technology utilizes segmented procedural material maps to represent appearances, supporting high-quality rendering and offering significant flexibility in editing. Leveraging a pre-trained 2D diffusion model, MaPa bridges the gap between text descriptions and material maps without requiring a large amount of paired data. The technology decomposes shapes into multiple parts and employs a control-segment diffusion model to synthesize 2D images aligned with the mesh segments, initializing the material map parameters. These are then refined through a differentiable rendering module to produce materials that match the textual description. Extensive experiments demonstrate that MaPa outperforms existing techniques in terms of realism, resolution, and editability.

URHand is the first universal light-aware hand model that can generalize across different viewpoints, poses, lighting conditions, and identities. The model can be personalized with a few shots from a mobile phone and render realistically under new lighting conditions. Based on neural network multi-view hand images and light, we build a strong universal light prior. We propose a neural renderer that utilizes a spatially-varying linear illumination model and physically-motivated shadows as input features. By removing non-linear activations and biases, our specifically designed illumination model explicitly maintains the linearity of light transport. We also introduce joint learning of physically-based models and neural light models, further enhancing the fidelity and generalization performance. Extensive experiments demonstrate that our method outperforms existing methods in both quality and generalization ability. We also demonstrate how to achieve rapid personalization for unseen identities from mobile phone shots.