# Computational Efficiency

1.58-bit FLUX is an advanced text-to-image generation model that employs 1.58-bit weights (values of {-1, 0, +1}) to quantify the FLUX.1-dev model while maintaining comparable performance for generating 1024x1024 images. This method does not require access to image data and relies entirely on the self-supervision of the FLUX.1-dev model. Additionally, a custom kernel has been developed to optimize 1.58-bit operations, achieving a 7.7x reduction in model storage, a 5.1x decrease in inference memory, and improved inference latency. Extensive evaluations in GenEval and T2I Compbench benchmarks show that 1.58-bit FLUX significantly enhances computational efficiency while maintaining generation quality.

SliceGPT is a new post-training sparsity approach that reduces the network's embedding dimension by replacing each weight matrix with a smaller (dense) matrix. Through extensive experiments, we demonstrate that SliceGPT can remove up to 25% of the model parameters (including embeddings) from LLAMA2-70B, OPT 66B, and Phi-2 models while maintaining 99%, 99%, and 90% of the zero-shot task performance, respectively. Our sliced models run on fewer GPUs and execute faster without any additional code optimizations: on a 24GB consumer-grade GPU, we reduce the total inference computation of LLAMA2-70B to 64% of the dense model; on a 40GB A100 GPU, we reduce it to 66%. We provide a new insight into the computational invariance in transformer networks, which makes SliceGPT possible. We hope it can inspire and promote new avenues for reducing memory and computational requirements of pre-trained models in the future.