Does Generative Face Completion Help Face Recognition?
Introduction
Most of the face recognition systems trained on a closed set of identities are often quite brittle to partial occlusions of the face. Some of the common occlusions we see in the wild are sunglasses, caps/hats, hands, or even scarves. Since most face biometric systems heavily depend on identification, a malicious actor could use one of these commonly found occlusion articles to evade detection or cause significant degradation of performance.
The biometric and vision communities have made significant efforts to develop robust occlusion-resistant face analysis algorithms such as landmark detectors resilient to partial occlusions, or face detection networks with occlusion-aware loss functions. However, less emphasis has been given to studying the impact of occlusion on recognition systems. Unlike some of the previous works, we take an orthogonal approach to handle face occlusions i.e., instead of optimizing the face recognition network with auxiliary objectives for occlusion robustness, we leverage an in-painting network to do a generative face completion followed by face recognition. Our goal with this research was to specifically study the impact of natural-looking occlusions on face recognition systems and try to quantify performance gained if the occluded area was in-painted with a learned prior.
To measure the effect of face completion on recognition and control the side effects of face obstruction, we employ a synthetic yet realistic data rendering mechanism to add occlusions to a face. With this pipeline we can generate realistic looking occlusions on non-occluded faces to create a new dataset for benchmarking algorithms. We also evaluate different convolutional gating schemes and propose a multi-discriminator adversarial network for realistic generative face completion. Based on the empirical results from the paper, face completion seems to be beneficial for a biometric system, assuming the occlusion can be detected effectively.
