%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
Cogagent
Overview :
CogAgent is a GUI agent based on visual language models (VLM) that facilitates bilingual (Chinese and English) cloud interaction through screenshots and natural language. CogAgent has made significant advancements in GUI perception, inference prediction accuracy, operational space integrity, and task generalization. The model has been applied in ZhipuAI's GLM-PC product, with the aim of aiding researchers and developers in advancing the research and application of GUI agents based on visual language models.
Target Users :
The target audience includes researchers and developers, particularly those seeking efficient solutions in the fields of GUI automation, visual language models, and natural language processing. The advanced technology provided by CogAgent can assist them in developing and researching GUI agents based on visual language models, advancing the development and application of related technologies.
Use Cases
Researchers conducting experiments on GUI perception and inference prediction using the CogAgent model.
Developers utilizing CogAgent to automate operations in desktop applications.
Businesses employing the CogAgent model to optimize customer service processes, enhancing efficiency through automated GUI operations.
Features
Supports bilingual (Chinese and English) cloud interaction via screenshots and natural language.
Demonstrates significant advantages in GUI perception, inference prediction accuracy, operational space integrity, and task generalization.
The CogAgent-9B-20241220 model is based on GLM-4V-9B, a bilingual open-source VLM foundational model.
Supports multi-stage training and policy improvement to achieve accuracy in GUI perception and inference prediction.
Model outputs follow strict formatting and are returned as strings; JSON output is not supported.
Does not support continuous dialogue but allows for sequential execution of history.
Requires images as input; pure text conversations cannot fulfill GUI agent tasks.
How to Use
1. Ensure that Python 3.10.16 or higher is installed and install the dependencies listed in requirements.txt.
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%230080ff;%20}%20%3c/style%3e%3c/defs%3e%3cpath%20class='st0'%20d='M16.2,11.1c.4.5.4,1.2,0,1.8l-4.7,7.1h-3.8l5.3-8L7.6,4h3.8l4.7,7.1Z'/%3e%3c/svg%3e)
2. Run the model using appropriate command-line parameters based on the desired output format and platform.
3. Provide the required input images to the model and receive the output containing operational instructions.
4. If the model returns results with bounding boxes, the output will include images indicating the execution areas.
5. Specify the output image's save location using the output image path parameter.
6. Adjust model parameters as necessary, such as maximum length and number of results returned.
7. For online web demonstrations, run web_demo.py and specify relevant parameters for interactive inference.
8. Refer to the project documentation and technical blogs for the model to gain in-depth knowledge of its usage and optimization.
Featured AI Tools
Gemini
Gemini is the latest generation of AI system developed by Google DeepMind. It excels in multimodal reasoning, enabling seamless interaction between text, images, videos, audio, and code. Gemini surpasses previous models in language understanding, reasoning, mathematics, programming, and other fields, becoming one of the most powerful AI systems to date. It comes in three different scales to meet various needs from edge computing to cloud computing. Gemini can be widely applied in creative design, writing assistance, question answering, code generation, and more.
AI Model
11.4M
Chinese Picks
Liblibai
LiblibAI is a leading Chinese AI creative platform offering powerful AI creative tools to help creators bring their imagination to life. The platform provides a vast library of free AI creative models, allowing users to search and utilize these models for image, text, and audio creations. Users can also train their own AI models on the platform. Focused on the diverse needs of creators, LiblibAI is committed to creating inclusive conditions and serving the creative industry, ensuring that everyone can enjoy the joy of creation.
AI Model
6.9M