# Multi-Modal

VideoLLaMA2-7B-16F-Base is a large video language model developed by the DAMO-NLP-SG team, focusing on Visual Question Answering (VQA) and video subtitling generation. Combining advanced space-time modeling and audio understanding capabilities, it provides strong support for multi-modal video content analysis. It demonstrates excellent performance in visual question answering and video subtitling generation tasks, capable of handling complex video content and generating accurate descriptions and answers.

Qmedia is a multimedia AI content search engine that supports text/image and short video material search. It provides content integration and cross-modal RAG (Retrieval-Augmented Generation) content answering, with advantages such as local deployment and multi-modal model services.

Developed by Professor Jia Jiayin's team at the Chinese University of Hong Kong, Mini-Gemini is a multi-modal model with precise image understanding capabilities and high-quality training data. Combining image reasoning and generation, it offers versions of different scales, with performance comparable to GPT-4 and DALLE3. Mini-Gemini utilizes Gemini's visual dual-branch information mining method and SDXL technology. It encodes images through convolutional networks and leverages the Attention mechanism to extract information, simultaneously connecting the two models by incorporating LLM for text generation.

Mobile-Agent is an autonomous multi-modal mobile device agent that leverages Multi-Modal Large Language Model (MLLM) technology. Firstly, it utilizes visual perception tools to accurately recognize and locate visual and textual elements on the front-end interface of applications. Based on the perceived visual environment, it autonomously plans and decomposes complex operational tasks and navigates mobile applications through step-by-step operations. Unlike previous solutions that relied on application-specific XML files or mobile system metadata, Mobile-Agent's vision-centric approach offers greater adaptability in various mobile operational environments, eliminating the need for customization to specific systems. To evaluate the performance of Mobile-Agent, we introduced Mobile-Eval, a benchmark for evaluating mobile device operations. Based on Mobile-Eval, we conducted a comprehensive evaluation of Mobile-Agent. Experimental results show that Mobile-Agent achieved significant accuracy and completion rates. Even with challenging instructions, such as multi-app operations, Mobile-Agent was still able to fulfill the requirements.

UniVG is a unified multi-modal video generation system that can handle various video generation tasks, including text and image modalities. By introducing multi-condition cross-attention and biased Gaussian noise, it achieves both high-freedom and low-freedom video generation. On the public academic benchmark MSR-VTT, it achieved the lowest Fréchet video distance (FVD), surpassing the performance of current open-source methods in human evaluation, and comparable to the current closed-source method Gen2.

Unified-IO 2 is a unified multi-modal generation model that can understand and generate images, text, audio, and actions. It utilizes a single encoder-decoder Transformer model to process inputs and outputs of different modalities (images, text, audio, actions, etc.) as representations within a shared semantic space. This model is trained from scratch on large-scale multi-modal pre-training data, using multi-modal denoising objectives for optimization. To learn a wide range of skills, the model is further fine-tuned on 120 existing datasets, which include prompts and data augmentation. Unified-IO 2 achieves state-of-the-art performance on the GRIT benchmark, achieving strong results across 30+ benchmarks, including image generation and understanding, text understanding, video and audio understanding, and robotics manipulation.

DevMind AI seamlessly integrates the reasoning capabilities of various models, including text, image, video, audio, and code, helping you develop like a pro! DevMind AI empowers your projects with AI functionalities.

Video Language Planning (VLP) is an algorithm that, through training visual language models and text-to-video models, achieves complex, long-term visual planning. VLP takes long-term task instructions and current image observations as input and outputs a detailed multi-modal (video and language) plan describing how to complete the final task. VLP can generate long-term video plans in various robotics domains, from multi-object re-arrangement to multi-camera dual-arm dexterous manipulation. The generated video plans can be converted into real robot actions through goal-conditioned policy. Experiments demonstrate that VLP significantly improves the success rate of long-term tasks compared to previous methods.

SEED is a large-scale pre-trained model that, through pre-training and guided fine-tuning on interwoven text and visual data, demonstrates outstanding performance in a wide range of multi-modal understanding and generation tasks. SEED also possesses emerging combinatorial capabilities, such as multi-turn contextual multi-modal generation, much like your AI assistant. SEED also includes SEED Tokenizer v1 and SEED Tokenizer v2, which can convert text into images.

Cognitiev PRO is an AI assistant powered by advanced GPT-4 technology, featuring safety, privacy, multi-platform compatibility, and multi-modality. It boasts 26 super-chat modes, each showcasing a unique AI application persona. Whether you need to enhance your coding and debugging skills or analyze art and code, Cognitiev PRO can meet your needs. Purchase Cognitiev PRO and unlock endless possibilities!

Pangu Model is an AI solution launched by Huawei Cloud. It utilizes multiple models, including NLP large models, CV large models, multi-modal large models, predictive large models, and scientific computing large models, to achieve various functions such as dialogue question answering, image recognition, multi-modal processing, predictive analysis, and scientific computing. Pangu Model boasts high adaptability, efficient labeling, and accurate controllability, making it widely applicable across industries. For more details, please visit the official website.

Gen-2 is a multi-modal artificial intelligence system that can generate new videos based on text, images, or video clips. It can apply the composition and style of images or textual prompts to the structure of source videos (Video to Video), or achieve this solely using text (Text to Video). It's like filming entirely new content without actually shooting anything. Gen-2 offers various modes to transform any image, video clip, or text prompt into captivating video works.