GASQ: Hardware-Friendly Gradient Distribution-Aware Split Quantization for Low-Bit CNN Training

As the demand for personalized AI models continues to rise and the importance of privacy protection grows, there has been increasing interest in efficiently training convolutional neural networks (CNNs) on mobile and edge devices. Since backward propagation (BP) requires significantly more computational resources and memory usage than forward propagation, low-bit quantization presents greater potential for improving the efficiency of CNN training. However, the variability and specificity of gradient distribution during BP make gradient quantization particularly challenging. Existing studies attempt to mitigate this issue through additional computations, but they often lead to increased hardware complexity. To address this, we propose a hardware-efficient INT8 quantization method, gradient distribution-aware split quantization (GASQ), which is robust to gradient quantization errors. GASQ employs distinct scale factors for small and large magnitude gradients, effectively capturing the gradient distribution, which is predominantly centered around zero yet spans a broad range. This approach maintains low hardware complexity while achieving minimal quantization error. The proposed method demonstrates an average 0.27% performance improvement over full-precision models on the ImageNet dataset for classification tasks.