This GitHub repository contains training and inference code to replicate our winning solution in the AI Mathematics Olympic (AIMO) Progress Prize 1. Our solution consists of four main parts: a recipe for fine-tuning DeepSeekMath-Base 7B for use in solving math problems using Tool Integrated Reasoning (TIR); two high-quality datasets of about 10 million math questions and solutions; an algorithm for generating solution candidates with coding execution feedback (SC-TIR); and four carefully selected validation sets from AMC, AIME, and MATH to guide model selection and avoid overfitting the public leaderboard.

HippoRAG is a novel Retriever-Augmented Generation (RAG) framework inspired by human long-term memory, enabling Large Language Models (LLMs) to continuously integrate knowledge across external documents. Experiments demonstrate that HippoRAG can provide the capabilities of RAG systems, typically requiring expensive and high-latency iterative LLM pipelines, at a lower computational cost.

Large language models increasingly rely on distributed techniques for training and inference. These techniques necessitate communication between devices, and as the number of devices increases, this can degrade scaling efficiency. While some distributed techniques can overlap communication to hide independent computation, techniques like tensor parallelism (TP) inherently serialize communication with model execution. One way to hide this serialized communication is to interweave it with producer operations (data generation) in a fine-grained manner. However, implementing this fine-grained communication and computation interleaving in software can be challenging. Furthermore, like any concurrent execution, it requires sharing computational and memory resources between computation and communication, leading to resource contention and decreased overlap efficiency. To overcome these challenges, we propose T3, which uses hardware-software co-design to transparently overlap serialized communication while minimizing resource contention with computation. T3, through simple configuration of producer output address spaces, transparently fuses producer operations and subsequent communication, requiring minimal software changes. At the hardware level, T3 incorporates lightweight tracking and triggering mechanisms to orchestrate producer computation and communication. It further leverages enhanced compute memory for computation related to communication. Consequently, T3 reduces resource contention and effectively overlaps serialized communication with computation. For important Transformer models like T-NLG, T3 achieves a geometric mean speedup of 30% (up to 47%) for communication-intensive sublayers and a geometric mean reduction of 22% (up to 36%) in data movement. Furthermore, T3's benefits persist as models scale: achieving a geometric mean speedup of 29% for sublayers in the 500B parameter sim model, PALM, and MT-NLG.

Zero Bubble Pipeline Parallelism is a crucial component of large-scale distributed training, and its efficiency is affected by pipeline bubbles. We introduce a scheduling strategy that successfully achieves zero pipeline bubbles under synchronous training semantics. The core idea behind this improvement is to divide backward calculation into two parts: one part calculates the gradients of the input, and the other part calculates the gradients of the parameters. Based on this idea, we manually designed novel pipeline scheduling, which significantly outperforms benchmark methods. We further developed an algorithm that automatically finds the optimal scheduling based on specific model configuration and memory constraints. Furthermore, to truly achieve zero bubbles, we introduce a novel technique that bypasses synchronization during optimizer steps. Experimental evaluation demonstrates that our method achieves up to 23% higher throughput than the 1F1B schedule under similar memory constraints. This number can further increase to 31% when memory constraints are relaxed. We believe our results mark an important step towards realizing the potential of pipeline parallelism.

ReFT is a simple yet effective method for enhancing the reasoning capabilities of large language models (LLMs). It first preheats the model through supervised fine-tuning (SFT), and then further fine-tunes the model using online reinforcement learning, specifically the PPO algorithm presented in this paper. ReFT significantly outperforms SFT by automatically sampling a large number of reasoning paths for a given problem and naturally deriving rewards from the true answers. ReFT's performance can be further improved by combining reasoning strategies (such as majority voting and re-ranking). It's noteworthy that ReFT achieves improvements by learning from the same training questions as SFT, without relying on additional or enhanced training questions. This demonstrates ReFT's stronger generalization ability.

This is an efficient LLM inference solution implemented on Intel GPUs. By simplifying the LLM decoder layer, utilizing segment KV caching strategies, and implementing a custom Scaled-Dot-Product-Attention kernel, this solution achieves up to 7x lower token latency and 27x higher throughput on Intel GPUs compared to the standard HuggingFace implementation. For detailed features, advantages, pricing, and positioning information, please refer to the official website.

The Intel? Gaudi? 3 AI Accelerator is a high-performance artificial intelligence accelerator launched by Intel, built on the efficient Intel? Gaudi? platform. It boasts outstanding MLPerf benchmark performance and is designed to handle demanding training and inference tasks. The accelerator supports AI applications such as large language models, multimodal models, and enterprise RAG in data centers or the cloud, operating on your existing Ethernet infrastructure. Whether you need a single accelerator or thousands, Intel Gaudi 3 can play a crucial role in your AI success.

This GitHub repository contains training and inference code to replicate our winning solution in the AI Mathematics Olympic (AIMO) Progress Prize 1. Our solution consists of four main parts: a recipe for fine-tuning DeepSeekMath-Base 7B for use in solving math problems using Tool Integrated Reasoning (TIR); two high-quality datasets of about 10 million math questions and solutions; an algorithm for generating solution candidates with coding execution feedback (SC-TIR); and four carefully selected validation sets from AMC, AIME, and MATH to guide model selection and avoid overfitting the public leaderboard.

HippoRAG is a novel Retriever-Augmented Generation (RAG) framework inspired by human long-term memory, enabling Large Language Models (LLMs) to continuously integrate knowledge across external documents. Experiments demonstrate that HippoRAG can provide the capabilities of RAG systems, typically requiring expensive and high-latency iterative LLM pipelines, at a lower computational cost.

Large language models increasingly rely on distributed techniques for training and inference. These techniques necessitate communication between devices, and as the number of devices increases, this can degrade scaling efficiency. While some distributed techniques can overlap communication to hide independent computation, techniques like tensor parallelism (TP) inherently serialize communication with model execution. One way to hide this serialized communication is to interweave it with producer operations (data generation) in a fine-grained manner. However, implementing this fine-grained communication and computation interleaving in software can be challenging. Furthermore, like any concurrent execution, it requires sharing computational and memory resources between computation and communication, leading to resource contention and decreased overlap efficiency. To overcome these challenges, we propose T3, which uses hardware-software co-design to transparently overlap serialized communication while minimizing resource contention with computation. T3, through simple configuration of producer output address spaces, transparently fuses producer operations and subsequent communication, requiring minimal software changes. At the hardware level, T3 incorporates lightweight tracking and triggering mechanisms to orchestrate producer computation and communication. It further leverages enhanced compute memory for computation related to communication. Consequently, T3 reduces resource contention and effectively overlaps serialized communication with computation. For important Transformer models like T-NLG, T3 achieves a geometric mean speedup of 30% (up to 47%) for communication-intensive sublayers and a geometric mean reduction of 22% (up to 36%) in data movement. Furthermore, T3's benefits persist as models scale: achieving a geometric mean speedup of 29% for sublayers in the 500B parameter sim model, PALM, and MT-NLG.

Zero Bubble Pipeline Parallelism is a crucial component of large-scale distributed training, and its efficiency is affected by pipeline bubbles. We introduce a scheduling strategy that successfully achieves zero pipeline bubbles under synchronous training semantics. The core idea behind this improvement is to divide backward calculation into two parts: one part calculates the gradients of the input, and the other part calculates the gradients of the parameters. Based on this idea, we manually designed novel pipeline scheduling, which significantly outperforms benchmark methods. We further developed an algorithm that automatically finds the optimal scheduling based on specific model configuration and memory constraints. Furthermore, to truly achieve zero bubbles, we introduce a novel technique that bypasses synchronization during optimizer steps. Experimental evaluation demonstrates that our method achieves up to 23% higher throughput than the 1F1B schedule under similar memory constraints. This number can further increase to 31% when memory constraints are relaxed. We believe our results mark an important step towards realizing the potential of pipeline parallelism.

ReFT is a simple yet effective method for enhancing the reasoning capabilities of large language models (LLMs). It first preheats the model through supervised fine-tuning (SFT), and then further fine-tunes the model using online reinforcement learning, specifically the PPO algorithm presented in this paper. ReFT significantly outperforms SFT by automatically sampling a large number of reasoning paths for a given problem and naturally deriving rewards from the true answers. ReFT's performance can be further improved by combining reasoning strategies (such as majority voting and re-ranking). It's noteworthy that ReFT achieves improvements by learning from the same training questions as SFT, without relying on additional or enhanced training questions. This demonstrates ReFT's stronger generalization ability.

This is an efficient LLM inference solution implemented on Intel GPUs. By simplifying the LLM decoder layer, utilizing segment KV caching strategies, and implementing a custom Scaled-Dot-Product-Attention kernel, this solution achieves up to 7x lower token latency and 27x higher throughput on Intel GPUs compared to the standard HuggingFace implementation. For detailed features, advantages, pricing, and positioning information, please refer to the official website.