Closit: Clipped-Regime Posit Quantization for Edge-Friendly Vision Transformers

Despite their outstanding performance, vision transformers (ViTs) pose significant challenges for deployment on resource-constrained edge devices due to their large number of parameters. Consequently, substantial research efforts have focused on reducing memory footprint and computational complexity through model compression techniques such as post-training quantization (PTQ). However, due to the inherent outlier characteristics of ViT, integer-based PTQ with limited representational range often suffers from significant quantization projection errors, leading to considerable performance degradation. While the Posit format offers a wide dynamic range to handle outliers, its variable-length decoding incurs high hardware overhead and does not align well with the skewed data distribution of ViT. To address these limitations, this paper proposes the clipped regime posit (Closit) format. Closit effectively reduces hardware overhead by simplifying the decoding process through a regime clipping technique. It also applies a regime remapping strategy tailored to ViT distributions, enabling more effective dynamic precision allocation. The Closit decoder saves 46.23% in area and 61.89% in power consumption compared to the conventional Posit decoder. Furthermore, Closit consistently outperforms existing formats across ViT models. It incurs only a 0.01% accuracy drop on ViT-B, highlighting its practicality as a balanced solution in terms of both accuracy and hardware efficiency.