Mask-Soft Filter Pruning for Lightweight CNN Inference

Pruning is a network compression and acceleration technique to reduce the computation and memory footprint required by Convolutional Neural Networks (CNNs), and various pruning techniques have been proposed by many researchers. Weight pruning (i.e., Unstructured pruning) can reduce many parameters by removing redundant weights, but it requires special software or hardware structure to actually accelerate the neural networks in the GPU environment. On the other hand, filter pruning to remove the filters itself does not require any special software or hardware structure, and consequently, it enables the actual acceleration of CNN in the GPU environment. Inspired by the previous research, soft filter pruning, which prunes the filters in a soft manner, this paper proposes Mask-Soft Filter Pruning(M-SFP) method. The proposed M-SFP is a pruning technique that can preserve weight parameters without zeroing out by masking the output feature maps. By applying the proposed technique to ResNet on CIFAR-10 and CIFAR-100 datasets, more than 40% reduction of BFLOPs can be achieved with only an acceptable accuracy drop of 0.17%.