

Universal Manipulation Interface
Overview :
The Universal Manipulation Interface (UMI) is a data collection and strategy learning framework that enables direct transfer of skills from human demonstrations on-site to deployable robot strategies. UMI combines handheld grips with meticulous interface design to achieve portable, low-cost, and informative data collection, used for challenging two-handed and dynamic operation demonstrations. To promote deployable strategy learning, UMI incorporates well-designed strategy interfaces with real-time delay matching and relative trajectory action representation. Thus, the learning strategies are hardware-agnostic and can be deployed on multiple robot platforms. Equipped with these features, the UMI framework opens up new capabilities for robot manipulation, allowing for generalized dynamic, two-handed, precise, and long-duration behaviors, achievable with zero adjustments. We have demonstrated the versatility and effectiveness of UMI through comprehensive real-world experiments, where UMI strategies trained with various human demonstrations achieved zero-shot generalization in new environments and with new objects.
Target Users :
["Robot skill learning","Handheld devices with external sensors","Human-robot interaction interface design"]
Use Cases
Collecting various daily actions using UMI, such as throwing a ball, folding clothes, washing dishes, etc.
Deploy trained strategies directly on different robot platforms without calibration
Using CLIP pre-trained ViT as a visual encoder to make strategies more responsive to changes
Features
Portable data collection, ready in 2 minutes
Camera-based action representation, no calibration required, robustness
Fast data collection, each demonstration in 30 seconds
Zero-shot generalization, deployable in new environments
Featured AI Tools

Openui
Building UI components is often tedious work. OpenUI aims to make this process fun, quick, and flexible. This is the tool we use at W&B to test and prototype the next generation of tools, built on top of LLMs to create powerful applications. You can describe your UI with imagination, and then see the rendering effect in real time. You can request changes, and convert HTML to React, Svelte, Web Components, and more. Think of it as an open-source and less polished version of a V0.
AI Development Assistant
757.9K

Opendevin
OpenDevin is an open-source project aiming to replicate, enhance, and innovate Devin—an autonomous AI software engineer capable of executing complex engineering tasks and actively collaborating with users on software development projects. Through the power of the open-source community, the project explores and expands Devin's capabilities, identifies its strengths and areas for improvement, thus guiding the advancement of open-source code models.
AI Development Assistant
595.1K