# Multi-task Learning

GR-2 is an advanced general-purpose robotic agent specifically designed for diverse and generalizable robotic operations. It undergoes extensive pre-training on a large dataset of internet videos to capture the dynamics of the world. This large-scale pre-training involves 38 million video clips and over 50 billion tags, enabling GR-2 to generalize across a wide range of robotic tasks and environments during subsequent policy learning. Subsequently, GR-2 is fine-tuned for video generation and action prediction using robotic trajectories. It demonstrates impressive multi-task learning capabilities, achieving an average success rate of 97.7% over more than 100 tasks. Moreover, GR-2 excels in new, previously unseen scenarios, including new backgrounds, environments, objects, and tasks. Notably, GR-2 efficiently scales with increasing model size, highlighting its potential for continuous growth and application.

Gemma-2-9b-it is a series of lightweight, state-of-the-art open models developed by Google, built upon the same research and technology as the Gemini model. These models are text-to-text decoder-only large language models, offered in English and suitable for a variety of text generation tasks, including question answering, summarization, and reasoning. Due to their relatively smaller size, they can be deployed in resource-limited environments such as laptops, desktops, or personal cloud infrastructure, making advanced AI models more accessible and fostering innovation.

Florence-2-large, developed by Microsoft, is an advanced vision foundation model that uses a prompt-based approach to handle a wide range of visual and visual-language tasks. The model can interpret simple text prompts to perform tasks such as image description, object detection, and segmentation. It is trained on the FLD-5B dataset, which contains 540 million images with 5.4 billion annotations, making it proficient in multi-task learning. Its sequence-to-sequence architecture enables it to perform well in both zero-shot and fine-tuning settings, proving to be a competitive vision foundation model.

Florence-2 is a novel visual foundation model that can handle various computer vision and vision-language tasks through a unified, prompt-based representation. Designed to accept text prompts as task instructions and generate expected results in textual format, whether it's image description, object detection, localization, or segmentation. This multi-task learning setup requires large-scale, high-quality annotated data. To this end, we jointly developed FLD-5B, which contains 5.4 billion comprehensive visual annotations across 126 million images, utilizing automated image annotation and model refinement iterative strategies. We employed a sequence-to-sequence structure to train Florence-2, enabling it to perform diverse and comprehensive visual tasks. Extensive evaluations demonstrate that Florence-2 is a powerful competitor within the visual foundation model landscape, exhibiting unprecedented zero-shot and fine-tuning capabilities.

Pile-T5 is a natural language processing model developed by EleutherAI. It builds upon the original T5 model, incorporating the Pile dataset and the LLAMA tokenizer during training to enhance its understanding of code-related tasks. This model has undergone training on 2 trillion tokens, twice the amount of training data used for the original T5. Pile-T5 demonstrates strong performance across various downstream tasks, particularly those involving code. EleutherAI also provides intermediate checkpoints, enabling researchers to study the model's evolution over time.

InternLM2, part of the 'Shusheng·Puyu 2.0' series, is a large-scale bilingual Chinese-English multilingual pre-trained language model. It boasts powerful capabilities in language understanding, natural language generation, multi-model reasoning, and code understanding. The model employs the Transformer architecture and has undergone extensive pretraining on a vast dataset, achieving industry-leading levels in long-text understanding, dialogue, mathematical calculations, and other areas. The series includes various scales, allowing users to select an appropriate model for downstream tasks, such as fine-tuning for specific applications or building chatbots.

Emu Edit is a multi-task image editing model that performs precise image editing by recognizing and generating tasks. It has made the latest technological breakthroughs in this field. Emu Edit's architecture is optimized for multi-task learning and trained on numerous tasks, including region-based editing, free-form editing, and computer vision tasks such as detection and segmentation. In addition, to more effectively handle these various tasks, we have introduced the concept of learned task embeddings to guide the generation process for accurately executing editing instructions. Our model, through multi-task training and the use of learned task embeddings, can significantly improve its ability to accurately execute editing instructions. Emu Edit also supports rapid adaptation to unseen tasks through task inversion for few-shot learning. In this process, we keep the model weights unchanged and only update the task embeddings to adapt to new tasks. Our experiments demonstrate that Emu Edit can quickly adapt to new tasks such as super-resolution and contour detection. This makes Emu Edit particularly advantageous for task inversion when labeled samples are limited or computational budgets are restricted. To support the strict and well-founded evaluation of instruction-based image editing models, we have also collected and publicly released a new benchmark dataset containing seven different image editing tasks: background modification, global image change, style modification, object removal, object addition, local modification, and color/texture modification. In addition, to allow for a fair comparison with Emu Edit, we also share Emu Edit's generation results on the dataset. Emu Edit 2023 Meta retains all copyrights