- Tips to fool facial recognition systems are fast becoming obsolete.
- New research using partial features is showing a high success rate.
- The future looks bleak for privacy advocates.
Facial recognition systems have exploded in popularity in the last couple of years. From filing for unemployment benefits to Uber identity checks, the technology is seeping into every aspect of our daily lives. But alongside increased security comes a dark side: invasion of privacy, unsanctioned government surveillance, and even jail time for wrongly identified criminals [1]. Multiple studies have addressed these issues by offering advice on how to thwart the cameras. But the tips and tricks you read about in 2020like wearing sunglasses and a maskmay no longer apply: the facial recognition developers are one step ahead of you.
Circumventing the System
One of the more common tips for circumventing the ID process is to occlude part of your face with a disposable face mask, glasses, or a combination of both. Last years National Institute of Standards and Technology (NIST) study showed that face masks were capable of fooling the best facial recognition systems between 5 and 50% of the time [2]. Thomas Smith, writing in a January 2021 post on Medium [3], ran an experiment that concluded you could beat the system by wearing opaque sunglasses and a mask. By adding a simple pair of sunglasses to my masked face, he states, I had effectively made myself invisible to the system. Another surprisingly simple approach was offered by researchers at Japanese IT giant NEC and at Ben-Gurion University of the Negev in Israel: apply natural looking makeup. They reported that conventional makeup techniques for contouring and changing the shape of the eyebrows may be enough to thwart the system [4]. However, these tips are fast becoming obsolete; Advances in partial facial recognitionwhere a system can make an identification using only the eyes and forehead, or similar small part of the faceare one step ahead of the game.
How the System is Fighting Back
Research into improvements in systems for partially occluded faces has been expressed as a dire need due to the widespread use of face maskswhich hide 75% of facial features. A quick keyword search of Google scholar for partial facial recognition shows that around 18,000 research articles have been published in 2021. Most of the research into partial facial recognition systems are for system improvements, which shouldnt come as a surprise: big industry and governments have deep pockets and a special interest in improving the systems. On the other hand, research into circumventing the systems is mostly academic or funded by non-profit entities.
Traditional Neural Network based models find it challenging to recognize faces that are partially occluded with hands, scarves, or other barriers. But Convolutional Neural Networks (CNNs) are highly capable of learning features and accurately identifying the image, even when the only fully visible part of the face is the eyes. Adding gradient descent optimization to the CNN approach has resulted in a better result for many researchers, making advances in identification from other features: eyebrows, cheeks, or foreheads. As a result, facial recognition systems research using CNNs is booming.
Highlights from Recent Studies
Panasonic was one of the first companies to crack the mask problem [5], and other companies quickly followed suit this year, including California-based Sensory, China-based companies Hanwang and SenseTime, and UK-based Facewatch [6]. More recent studies have developed CNN-based systems that can correctly identify who is behind the sunglasses, scarf, or facemask, with up to almost 99% sensitivity (a measure of true positives) [7]. The research was tested and compared on the AR face dataset, which contains images where eyes are covered by glasses, mouths are covered by scarves, and sometimes faces are occluded by both at the same time. High accuracy has been reported in numerous other studies for low-lighting conditions and noise [8], or in real-life situations like busy streets.
While results from studies using just one feature (e.g., cheeks, eyebrows, or forehead) are lukewarm, achieving less than 50% success, that doesnt mean we should be complacent. Tradition full-face recognition systems have improved 50-fold over the last six years [9], making it just a matter of time before an exposed eyebrow can lead to a positive ID. Fox Cahn, from the Surveillance Technology Oversight Project, stated in a Vox article [7] that he doubted sunglasses or other trickery will work for long. Well get to the point where the cameras are so prolific and the technology is so powerful, he said, that anything short of a full bodysuit is going to be trackable.
References
Top Image: Adobe Creative Cloud
[1] A Flawed Facial-Recognition System Sent This Man to Jail
[2] NIST Launches Studies into Masks Effect on Face Recognition Software
[3] This Simple Technique Made Me Invisible to Two Major Facial Recogni…
[4] Dodging Attack Using Carefully Crafted Natural Makeup
[5] Coronavirus is the first big test for futuristic tech that can prev…
[6] Masks can fool facial recognition systems, but the algorithms are l…
[7] FRS-OCC: Face Recognition System for Surveillance Based on Occlusio…
[8] Detection of Partially Occluded Faces Using Convolutional Neural Ne…
[9] Facial recognition: top 7 trends (tech, vendors, markets, use cases…
