🤖BITCOS Breaks 1.58-bit Barrier for Ternary LLMs
New Layout Shaves 0.105 Bits Per Weight
TL;DR
BITCOS, a novel layout for ternary large language models, slashes storage costs by up to 1.28 times. It breaks the 1.585-bit barrier, offering up to 1.485 bits per weight on sparse models. This could be a game-changer for LLM deployment.
BITCOS, a new layout for ternary large language models, slashes storage costs by up to 1.28 times. It breaks the 1.585-bit barrier, offering up to 1.485 bits per weight on sparse models. This could be a game-changer for LLM deployment, especially for teams running resource-intensive models. If you're working with sparse ternary LLMs, this layout could save you significant storage space and improve inference speeds. The paper, submitted to arXiv on September 14, 2026, details how BITCOS reaches these gains through a dense presence bitmap and a compacted sign vector, optimized for modern processors and GPUs.
Key Points
BITCOS layout breaks the 1.585-bit barrier, reaching 1.485 bits per weight on sparse models.
BITCOS stores weights more compactly than five-trit packing in 26 of 29 tested models.
BITCOS improves decode throughput by up to 1.18 times on CPUs and 1.27 times on GPUs.
The layout is optimized for AVX-512, AVX2, and Intel Xe2 GPUs, enhancing efficiency.
Authors Evangelos Georganas, Alexander Heinecke, and Pradeep Dubey submitted the paper on September 14, 2026.
Why It Matters
BITCOS breaks the 1.58-bit barrier for ternary LLMs, offering up to 1.28 times storage savings. This could be a big win for teams with sparse models, reducing storage costs and improving inference speeds. The optimized layout for modern processors and GPUs means faster, more efficient model deployment.
Comments
Be the first to comment
Enjoyed this article?
Get it daily. 7am. Free. Reads in 5 minutes.
Join 3,482 builders reading daily.