1.58 bits per parameter

On October 17, 2024, Microsoft introduced BitNet.cpp, an inference framework designed to run 1-bit quantized Giant Language Fashions (LLMs). BitNet.cpp is a major progress in Gen AI, enabling the deployment of 1-bit LLMs effectively on commonplace CPUs, with out...