DETR with Additional Object Instance-Specific Features for Encoder

This paper focuses on the process of developing from a convolutional neural network (CNN)-based target detection method to a transformer-based DETR paradigm-based target detection method. DETR adopts a Transformer-based end-to-end detection method, and it does not use the traditional anchor box and non-maximum suppression by transforming target detection into a set prediction problem. DETR has shown competitive results on public datasets and brought new ideas and methods to the field of object detection. We observed that DETR and DETE-like models include backbone and encoder that have same effect on the image, that is, they both did the same feature extraction function. We propose to add additional embedding module, which represents the full class information, and establishes global attention between feature tokens to provide prior knowledge for the extractor.