ManiWAV
M
Maniwav
Overview :
ManiWAV is a research project aimed at learning robot manipulation skills from wild audio and visual data. It collects synchronized audio and visual feedback from human demonstrations and directly learns robot manipulation policies from these demonstrations through corresponding policy interfaces. The model demonstrates its system's capabilities through four contact-rich manipulation tasks, which require the robot to passively perceive contact events and patterns or actively perceive the material and state of object surfaces. Additionally, the system can generalize to unseen wild environments by learning from diverse wild human demonstrations.
Target Users :
ManiWAV is primarily targeted toward researchers and developers in the robotics and artificial intelligence fields. They can leverage ManiWAV to enhance robot manipulation skills in complex environments, particularly when visual information is unclear or incomplete, by augmenting the robot's perception capabilities through auditory information.
Total Visits: 400
Website Views : 45.3K
Use Cases
Using the ManiWAV system to wipe graphics off a whiteboard
Utilizing ManiWAV to flip pancakes in a frying pan
Using ManiWAV to transfer dice from one cup to another
Using ManiWAV to secure wires with Velcro tape
Features
Provides rich interaction and object attribute information through audio signals
Uses 'ear-in-hand' data collection devices to synchronously collect audio and visual feedback
Directly learns robot manipulation policies from human demonstrations
Demonstrates system capabilities in four contact-rich manipulation tasks
Learns through diverse wild human demonstrations to achieve generalization to unseen environments
How to Use
1. Install and configure ManiWAV data collection devices, ensuring synchronized audio and visual feedback.
2. Collect human demonstration data, including audio signals of contact events and object surface materials.
3. Train robot manipulation policies using the collected data.
4. Test and validate the effectiveness of the policies in diverse wild environments.
5. Adjust the policies based on testing results to improve the robot's generalization capability.
6. Apply the trained policies to actual robot manipulation tasks.
AIbase
Empowering the Future, Your AI Solution Knowledge Base
© 2025AIbase