IoU-uniform R-CNN: Breaking through the limitations of RPN

• We reveals the importance of solving the limitations of RPN and our proposed IoU-uniform R-CNN can alleviate the IoU distribution imbalance and inadequate training samples by generating samples with uniform IoU distribution. • We improve the performance of IoU prediction branch by eliminating the feature offsets of RoIs at inference. • Our proposed method consistently obtains significant improvements over multiple state-of-the-art detectors. Specially, it achieves 2.4 AP improvement than Faster R-CNN (with ResNet 101-FPN backbone) on MS COCO dataset. Region Proposal Network (RPN) is the cornerstone of two-stage object detectors. It generates a sparse set of object proposals and alleviates the extrem foreground-background class imbalance problem during training. However, we find that the potential of the detector has not been fully exploited due to the IoU distribution imbalance and inadequate quantity of the training samples generated by RPN. With the increasing intersection over union (IoU), the exponentially smaller numbers of positive samples would lead to the distribution skewed towards lower IoUs, which hinders the optimization of detector at high IoU levels. In this paper, to break through the limitations of RPN, we propose IoU-Uniform R-CNN, a simple but effective method that directly generates training samples with uniform IoU distribution for the regression branch as well as the IoU prediction branch. Besides, we improve the performance of IoU prediction branch by eliminating the feature offsets of RoIs at inference, which helps the NMS procedure by preserving accurately localized bounding box. Extensive experiments on the PASCAL VOC and MS COCO dataset show the effectiveness of our method, as well as its compatibility and adaptivity to many object detection architectures. The code is made publicly available at https://github.com/zl1994/IoU-Uniform-R-CNN .