%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
EXAONE 3.5 2.4B Instruct GGUF
Overview :
EXAONE-3.5-2.4B-Instruct-GGUF is a series of bilingual (English and Korean) generative models fine-tuned for instructions, developed by LG AI Research, with parameter sizes ranging from 2.4B to 32B. These models support long context processing of up to 32K tokens, demonstrating state-of-the-art performance in real-world use cases and long context comprehension, while remaining competitive in the general domain compared to similarly sized recently released models. The significance of this model lies in its optimization for deployment on small or resource-constrained devices, while delivering powerful performance.
Target Users :
Designed for researchers and developers who need to deploy high-performance language models on resource-constrained devices, as well as application developers working on long text and multilingual text generation. This model is particularly suited for these users as it offers optimized deployment options and robust performance while supporting long context understanding and multilingual capabilities.
Use Cases
Researchers utilize the EXAONE-3.5-2.4B-Instruct-GGUF model for semantic understanding studies of long texts.
Developers leverage this model for real-time multilingual translation features on mobile devices.
Businesses employ the model to optimize automated response systems in customer service, enhancing efficiency and accuracy.
Features
Supports long context processing capabilities of up to 32K tokens.
Includes three different model sizes: 2.4B, 7.8B, and 32B, to meet various deployment needs.
Demonstrates state-of-the-art performance in real-world use cases.
Supports bilingual (English and Korean) text generation.
Fine-tuned for instructions, enabling better understanding and execution of directives.
Provides multiple quantized versions of the model to accommodate varying computational and storage needs.
Allows inference across various frameworks such as TensorRT-LLM, vLLM, SGLang, and more.
The text generated by the model does not reflect the views of LG AI Research, and LG AI Research actively works to mitigate potential risks associated with the model.
How to Use
1. Install llama.cpp; for specific installation instructions, refer to the llama.cpp GitHub repository.
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%230080ff;%20}%20%3c/style%3e%3c/defs%3e%3cpath%20class='st0'%20d='M16.2,11.1c.4.5.4,1.2,0,1.8l-4.7,7.1h-3.8l5.3-8L7.6,4h3.8l4.7,7.1Z'/%3e%3c/svg%3e)
2. Download the GGUF format file for the EXAONE 3.5 model.
3. Use the huggingface-cli tool to download the specified model file to your local directory.
4. Run the model using the llama-cli tool and set a system prompt, e.g., 'You are the EXAONE model from LG AI Research, a helpful assistant.'
5. Choose the appropriate quantized version of the model for deployment and inference as needed.
6. Deploy the model in supported frameworks like TensorRT-LLM, vLLM, etc., for practical applications.
7. Monitor the text generated by the model to ensure compliance with LG AI's ethical guidelines.
8. Further optimize the model's use and performance based on technical reports, blogs, and guidance available on GitHub.
Featured AI Tools
Gemini
Gemini is the latest generation of AI system developed by Google DeepMind. It excels in multimodal reasoning, enabling seamless interaction between text, images, videos, audio, and code. Gemini surpasses previous models in language understanding, reasoning, mathematics, programming, and other fields, becoming one of the most powerful AI systems to date. It comes in three different scales to meet various needs from edge computing to cloud computing. Gemini can be widely applied in creative design, writing assistance, question answering, code generation, and more.
AI Model
11.4M
Chinese Picks
Liblibai
LiblibAI is a leading Chinese AI creative platform offering powerful AI creative tools to help creators bring their imagination to life. The platform provides a vast library of free AI creative models, allowing users to search and utilize these models for image, text, and audio creations. Users can also train their own AI models on the platform. Focused on the diverse needs of creators, LiblibAI is committed to creating inclusive conditions and serving the creative industry, ensuring that everyone can enjoy the joy of creation.
AI Model
6.9M