MInference is an inference acceleration framework for long context large language models (LLMs). It leverages the dynamic sparse characteristics of LLMs' attention mechanisms, significantly enhancing the speed of pre-filling through static pattern recognition and online sparse index approximation computation. It achieves a tenfold increase in processing 1M contexts on a single A100 GPU while maintaining inference accuracy.

The Intel NPU Acceleration Library is designed to enhance the performance of deep learning and machine learning applications on Intel's Neural Processing Units (NPUs). It offers algorithms and tools optimized for Intel hardware, supports various deep learning frameworks, and significantly improves model inference speed and efficiency.

OpenDiT is an open-source project providing a high-performance implementation of Diffusion Transformer (DiT) based on Colossal-AI. It is designed to enhance the training and inference efficiency of DiT applications, including text-to-video and text-to-image generation. OpenDiT achieves performance improvements through the following technologies: * GPU acceleration up to 80% and 50% memory reduction; * Core optimizations including FlashAttention, Fused AdaLN, and Fused layernorm; * Mixed parallelism methods such as ZeRO, Gemini, and DDP, along with model sharding for ema models to further reduce memory costs; * FastSeq: A novel sequence parallelism method particularly suitable for workloads like DiT, where activations are large but parameters are small. Single-node sequence parallelism can save up to 48% in communication costs and break through the memory limit of a single GPU, reducing overall training and inference time; * Significant performance improvements can be achieved with minimal code modifications; * Users do not need to understand the implementation details of distributed training; * Complete text-to-image and text-to-video generation workflows; * Researchers and engineers can easily use and adapt our workflows to real-world applications without modifying the parallelism part; * Training on ImageNet for text-to-image generation and releasing checkpoints.

Deepchecks is a comprehensive open-source solution for continuous validation of AI and ML models and data. It comprises three components: Deepchecks Testing, CI & Testing Management, and Deepchecks Monitoring. Deepchecks Testing allows you to run both built-in and custom checks and suites for Tabular, NLP, and CV validation. CI & Testing Management enables collaborative testing results and iteration until the model is ready for production. Deepchecks Monitoring allows you to track and validate the behavior of deployed models in production. Deepchecks also includes a wide range of built-in checks for testing model performance, data distribution, data integrity, and more. Check results can be examined and collaborated on through visualization reports, code integration, and Deepchecks' dynamic UI. While Deepchecks is open-source, certain components of Deepchecks Monitoring require a commercial license.

MInference is an inference acceleration framework for long context large language models (LLMs). It leverages the dynamic sparse characteristics of LLMs' attention mechanisms, significantly enhancing the speed of pre-filling through static pattern recognition and online sparse index approximation computation. It achieves a tenfold increase in processing 1M contexts on a single A100 GPU while maintaining inference accuracy.

Powered by six revolutionary technologies, the NVIDIA Blackwell platform drives accelerated computing, enabling real-time AI generation and processing of language models with up to hundreds of billions of parameters, while reducing costs and energy consumption.

The Intel NPU Acceleration Library is designed to enhance the performance of deep learning and machine learning applications on Intel's Neural Processing Units (NPUs). It offers algorithms and tools optimized for Intel hardware, supports various deep learning frameworks, and significantly improves model inference speed and efficiency.

OpenDiT is an open-source project providing a high-performance implementation of Diffusion Transformer (DiT) based on Colossal-AI. It is designed to enhance the training and inference efficiency of DiT applications, including text-to-video and text-to-image generation. OpenDiT achieves performance improvements through the following technologies: * GPU acceleration up to 80% and 50% memory reduction; * Core optimizations including FlashAttention, Fused AdaLN, and Fused layernorm; * Mixed parallelism methods such as ZeRO, Gemini, and DDP, along with model sharding for ema models to further reduce memory costs; * FastSeq: A novel sequence parallelism method particularly suitable for workloads like DiT, where activations are large but parameters are small. Single-node sequence parallelism can save up to 48% in communication costs and break through the memory limit of a single GPU, reducing overall training and inference time; * Significant performance improvements can be achieved with minimal code modifications; * Users do not need to understand the implementation details of distributed training; * Complete text-to-image and text-to-video generation workflows; * Researchers and engineers can easily use and adapt our workflows to real-world applications without modifying the parallelism part; * Training on ImageNet for text-to-image generation and releasing checkpoints.

Deepchecks is a comprehensive open-source solution for continuous validation of AI and ML models and data. It comprises three components: Deepchecks Testing, CI & Testing Management, and Deepchecks Monitoring. Deepchecks Testing allows you to run both built-in and custom checks and suites for Tabular, NLP, and CV validation. CI & Testing Management enables collaborative testing results and iteration until the model is ready for production. Deepchecks Monitoring allows you to track and validate the behavior of deployed models in production. Deepchecks also includes a wide range of built-in checks for testing model performance, data distribution, data integrity, and more. Check results can be examined and collaborated on through visualization reports, code integration, and Deepchecks' dynamic UI. While Deepchecks is open-source, certain components of Deepchecks Monitoring require a commercial license.