Batch Transformer: Look for Attention in Batch

Facial expression recognition (FER) has received considerable attention in computer vision, with “in-the-wild” environments such as human-computer interaction. However, FER images contain uncertainties such as occlusion, low resolution, pose variation, illumination variation, and subjectivity, which include some expressions that do not match the target label. Consequently, minimal information is obtained from a noisy single image, making it unreliable. This can significantly degrade the performance of the FER task. To address this issue, we propose a batch transformer (BT), comprising the proposed class batch attention (CBA) module to prevent overfitting in noisy data and to extract trustworthy information by training on features reflected from several images in a batch, instead of information from a single image. We also propose multi-level attention (MLA) to prevent the overfitting of specific features by capturing the correlations between each level. In this paper, we present a batch transformer network (BTN) that combines the above proposals. The proposed BTN improves the performances of RAF-DB, FER2013, AffectNet 7 classes, and 8 classes by 0.33%, 1.37%, 0.11%, and 0.52% points, respectively, by integrating MLA and BT with the baseline POSTER V2. Additionally, the BTN enhances performance by 1.22%, 1.67%, 0.36%, and 0.40% points on occlusion-RAF-DB, Oulu-CASIA NIR Dark, Resolution-28 RAF-DB, and Pose-RAF-DB, respectively, as noisy FER datasets. Notably, BT and MLA achieve performance improvements using only 1.3 million additional parameters and 0.18 FLOPs of additional computation. These experimental results demonstrate that the BTN improves robustness to noisy data while ensuring memory-efficiency.