

Aimo Progress Prize
Overview :
This GitHub repository contains training and inference code to replicate our winning solution in the AI Mathematics Olympic (AIMO) Progress Prize 1. Our solution consists of four main parts: a recipe for fine-tuning DeepSeekMath-Base 7B for use in solving math problems using Tool Integrated Reasoning (TIR); two high-quality datasets of about 10 million math questions and solutions; an algorithm for generating solution candidates with coding execution feedback (SC-TIR); and four carefully selected validation sets from AMC, AIME, and MATH to guide model selection and avoid overfitting the public leaderboard.
Target Users :
The product is suitable for researchers and students in the fields of mathematics and computer science, particularly those interested in the application of AI in solving complex mathematical problems. It can help them understand how to use deep learning models to improve their ability to solve mathematical problems.
Use Cases
Researchers use the model to improve their ability to solve math competition problems.
Students use the model to learn and understand complex mathematical concepts.
Educators use the model as a teaching aid, helping students master problem-solving skills.
Features
Fine-tune the DeepSeekMath-Base 7B model to solve math problems
Train the model using two high-quality datasets of math questions and solutions
Use the Self-Concordant Decoding algorithm to generate solution candidates
Use validation sets from AMC, AIME, and MATH to guide model selection
Train the model using open-source libraries TRL, PyTorch, vLLM, and DeepSpeed
The model training is divided into two stages: CoT training and TIR training
How to Use
1. Create a Python virtual environment and activate it.
2. Install a specific version of PyTorch to ensure reproducibility.
3. Install other necessary package dependencies.
4. Install Flash Attention 2.
5. Log in to your Hugging Face account.
6. Install Git LFS to push the model to the Hugging Face Hub.
7. Conduct two-stage training according to the MuMath-Code recipe: CoT training and TIR training.
8. (Optional) After training, use AutoGPTQ to quantize the model to 8-bit.
Featured AI Tools

Screenshot To Code
Screenshot-to-code is a simple application that uses GPT-4 Vision to generate code and DALL-E 3 to generate similar images. The application has a React/Vite frontend and a FastAPI backend. You will need an OpenAI API key with access to the GPT-4 Vision API.
AI code generation
970.4K

Codegemma
CodeGemma is an advanced large language model released by Google, specializing in generating, understanding, and tracking instructions for code. It aims to provide global developers with high-quality code assistance tools. It includes a 2 billion parameter base model, a 7 billion parameter base model, and a 7 billion parameter model for guiding tracking, all optimized and fine-tuned for code development scenarios. It excels in various programming languages and possesses exceptional logical and mathematical reasoning abilities.
AI code generation
328.2K