Quantization in Machine Learning: Step-by-Step Guide & Math Demystified

Quantization is a MUST Step to Fine-Tune Large Language Models

Suppose you are fitting a large number of books into a small suitcase. You can’t take all of them, so you must decide which ones to bring and which to leave behind. This process of selecting and compressing data is quite similar to what we do in machine learning when we perform quantization.

Quantization is a technique for reducing the number of bits needed to represent data. This reduces the number of bits needed to compress models, making them faster and more efficient.

In this article, we’ll delve into the concept of quantization, its types, and how to perform it effectively.

· What is Quantization and Why is it Important?
∘ Why Quantization Matters
∘ You might be wondering, How?
· Types of Quantization
∘ Symmetric Quantization
∘ Asymmetric Quantization
· How to Perform Quantization
∘ Symmetric Quantization
∘ Asymmetric Quantization
· Conclusion
∘ Key Takeaways