%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
Aria UI
Overview :
Aria-UI is a large-scale multimodal model specifically designed for visual localization of GUI commands. It employs a purely visual approach without relying on auxiliary inputs, accommodating a variety of planning commands and generating diverse, high-quality command samples to adapt to different tasks. Aria-UI has set new records in both offline and online agent benchmarks, surpassing baselines that rely solely on visual inputs or AXTree.
Target Users :
Aria-UI is designed for digital agents and researchers who need to automate GUI tasks. By providing robust visual localization capabilities, it enhances the efficiency and accuracy of task automation, especially in scenarios involving complex GUIs and diverse commands.
Use Cases
Automate the task of stopping services by interpreting GUI commands and locating the stop service button.
Verify the color palette by visually locating the palette area within the GUI.
Enable iCloud photo features by identifying and interacting with the iCloud settings in the GUI.
Features
- ? Multi-format command understanding: Aria-UI can process a variety of localization commands, adapting to different formats and ensuring robust adaptability in dynamic environments or with various planning agents.
- ?? Context-aware localization: Aria-UI effectively utilizes historical inputs, whether in pure text or mixed formats, to enhance localization accuracy.
- ? Lightweight and fast: As a mixed expert model with 3.9 billion parameters activated per token, Aria-UI efficiently encodes GUI inputs of varying sizes and aspect ratios, supporting ultra-high resolutions.
- ?? Outstanding performance: Aria-UI ranked first and third in the AndroidWorld and OSWorld benchmarks, respectively, showcasing its exceptional performance.
How to Use
1. Visit the Aria-UI HF Space Demo page to experience the model's capabilities online.
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%230080ff;%20}%20%3c/style%3e%3c/defs%3e%3cpath%20class='st0'%20d='M16.2,11.1c.4.5.4,1.2,0,1.8l-4.7,7.1h-3.8l5.3-8L7.6,4h3.8l4.7,7.1Z'/%3e%3c/svg%3e)
2. Download and install the necessary Aria-UI datasets and model checkpoints for local use.
3. Read the Aria-UI paper and documentation to understand the model's functionality and usage.
4. Write or adjust localization commands according to specific GUI tasks to meet Aria-UI's input requirements.
5. Utilize the Aria-UI model for visual localization of the GUI and perform automation tasks.
6. Adjust and optimize model parameters as needed to enhance task execution accuracy and efficiency.
Featured AI Tools
Chinese Picks
Douyin Jicuo
Jicuo Workspace is an all-in-one intelligent creative production and management platform. It integrates various creative tools like video, text, and live streaming creation. Through the power of AI, it can significantly increase creative efficiency. Key features and advantages include:
1. **Video Creation:** Built-in AI video creation tools support intelligent scripting, digital human characters, and one-click video generation, allowing for the rapid creation of high-quality video content.
2. **Text Creation:** Provides intelligent text and product image generation tools, enabling the quick production of WeChat articles, product details, and other text-based content.
3. **Live Streaming Creation:** Supports AI-powered live streaming backgrounds and scripts, making it easy to create live streaming content for platforms like Douyin and Kuaishou. Jicuo is positioned as a creative assistant for newcomers and creative professionals, providing comprehensive creative production services at a reasonable price.
AI design tools
105.1M
Promeai
PromeAI is powered by a robust AI-driven design assistant and a vast library of controllable AIGC (C-AIGC) model styles. It enables you to effortlessly create stunning graphics, videos, and animations, making it an indispensable tool for architects, interior designers, product designers, and game & animation designers.
AI design tools
6.5M