Pluses and perils of face recognition

Built on computer programs that analyze images of human faces to identify them, face recognition technology has seen an amazing maturity in a relatively short amount of time. Law enforcement agencies use it to keep society safer, retailers to reduce the incidence of thefts, airports to enhance travelers’ comfort and security, and cell phone companies to provide customers with new layers of biometric security.

Face recognition technology has improved rapidly in recent years due to advances in Artificial Intelligence (AI) and Machine Learning (ML). Thanks to sufficient high-quality training data and processing power, computers can now detect, classify, and recognize faces with high accuracy and speed. Computerized identification of the human face has paved the way for the growing application of this technology in a wide range of areas.

In the wake of multifold rise in crimes and terrorist activities in recent decades, the need has been felt for a sophisticated security system for the identification of individuals in which face-based applications like face detection and face recognition can play a crucial role.

It is pertinent here to explore how this transformative tech has evolved over the years:

Evolution of face recognition technique

Early adopters of face recognition techniques include banks, event managers, forensic investigators, military professionals, and law enforcement agencies. Here are some significant milestones in the progress of face recognition technology over the past decades:

The origin of this tech can be traced back to the 1960s when Woodrow Wilson Bledsoe created first manual measurements of various facial features, including the eyes, nose, hairline, and mouth using electromagnetic pulses, which were inserted in a database. Subsequently, on receipt of a new photograph, the system was able to retrieve the image from the database that most closely resembled that individual.

In the 1970s, researchers Goldstein, Harmon, and Lesk established 21 points of facial measurement, including lip thickness and hair color, to identify faces automatically.

In 1980s, Kirby and Sirovich used around 100 points of facial measurement using linear algebra, paving the way for a low-dimensional representation of facial images.

In 1990s, Turk and Pentland invented first crude automatic face detection from images. Then the Defense Advanced Research Projects Agency (DARPA) and the National Institute of Standards and Technology (NIST) rolled out the Face Recognition Technology (FERET) program that involved creating a database of facial images. This database was upgraded in 2003 to include high-resolution 24-bit color versions of images.

In the 2010s, Facebook created image identity auto-tagging people whose faces may be featured in the photos that FB users update daily. Since then, a million photos are uploaded and tagged on FB using face recognition each day.

In 2011, the Panama Airport installed a first face recognition surveillance system to cut down on drug smuggling and organized crime. The system resulted in the apprehension of multiple Interpol suspects.

Significantly, in 2011, the body of al-Qaeda chief Osama bin Laden was identified after he was killed in a US raid, using face recognition. A visual ID was made; several of his photo comparisons and other facial recognitions were used, and a full biometric analysis of facial and body features was done to identify bin Laden.

In 2017, iPhone X became the world’s top-selling phone with face recognition access control that used for device security.

Currently, Japan plans to use face recognition to verify the identity of athletes at the 2020 Olympic Games, to be held in Tokyo in July-August.

Steps involved in face recognition

In human-computer vision analogy, a camera resembles the human eye, and the computer resembles the human brain. The camera captures the pictures of the world and further processing and interpretation are carried out on a computer. Different techniques may be employed for processing and analysis of captured images.

The human can easily identify that from where the one object ends, and then another starts in a single image. The edges of objects, color differences, and textures are used for identification in computers, which is called segmentation.

The computer can see any object, carry out its identification as well as the processing of the image in the same way as the human sight does, and also the appropriate output is generated. The image under observation is interpreted, and respective proper analysis and actions are carried out.

The track of a particular object is found from several frames of the image sequence. The velocity and distance of the particular object can be calculated as well as future path prediction is carried out.

The different matching points between the images, the positions of camera and orientation, the reconstruction of three-dimensional shapes in the scene can be found from several images of a stationary scene.

Broadly, the following steps are involved in face recognition technology:

Face Detection: A computer-aided algorithm pre-processes images captured by the camera. It leads to an easily recognizable clean image. Subsequently, localizing is carried out, and features of the face are extracted.

Face Recognition: The detected face image is processed and compared with the database of known persons; the extracted features are compared with stored features to find out the likelihood of a match. They are arranged in groups or classes as per the similar characteristic to recognize an individual.

Automatic verification of an individual’s identity is possible by one to one matching of an unknown person with face image in records for identification. The identity of an unknown person from the group of persons is carried out by encoding and comparing it with the database containing encoded images of known individuals.

Uses and applications of face recognition technique

Face recognition has found its importance in various fields, such as security validation and human-computer interaction. As a result, it has become a research hotspot in the areas of pattern recognition. The researchers are pursuing the successful implementation of a working facial recognition system in the field of computer vision. The exhaustive potential of this tech can be envisaged in the areas like border control and key lock mechanisms replacement.

Akin to the Internet, GPS, and many other technologies that have become part and parcel of various products today, face recognition’s roots are firmly planted in the defense and law enforcement sectors where it is useful in verification and identification of images.

Law enforcement agencies are widely using face recognition technology. Cameras at surveillance place capture images of all the objects in real-time, continuous, and, most importantly, without anybody’s notice.

Facial recognition surveillance identifies all individuals as they move about their daily life. While the ability to recognize individuals in real-time has become a reality, in Western countries like the United States, most people in a face recognition database of documented persons of interest are included due to a track record of prior offense/s. For instance, when retailers want to apprehend people attempting to steal from their stores, they rely on the photographs of these individuals, uploaded into a private face recognition database. As shoplifters are mostly compulsive offenders, an alert can direct in-store security to observe such individuals when they re-enter the shops. It leads to a reduced incidence of thefts and a much smaller chance of violence.

The London Metropolitan Police is using face recognition technology to find wanted criminals and missing persons. The technology has been deployed at “specific locations,” each with a “tailor-made watch list” of wanted persons, usually violent offenders.

India is headed for increasing the use of emerging technologies in law enforcement as police forces across the country plan to replace manual processes with tech-driven solutions. Engineering institutes are helping the police leverage AI, social media analytics, and image processing to identify criminals, for traffic management and to prevent terrorist activities.

Police forces in India are beginning to embrace technologies, and collaborating with digital leaders to fight crime. They have implemented the system of filing first information reports (FIRs) electronically to reduce human involvement by submitting them through apps, websites, or even an Internet of Things (IoT) device. The comprehensive strategy adopted by the police includes identifying and capturing digital and non-digital evidence using AI and ML, face recognition, and virtually predicting crimes. The initiatives have led to the building of a criminal database using AI-based human face detection (ABHED). As and when police officers spot suspicious characters, they are supposed to take out their smartphones, click a picture, and tap into the database to determine whether he/she is a criminal.

The technology may not be 100% reliable, but it makes the job of a police job easier. It has sown the seeds of a high-tech force that has access to photos, criminal activity, and physical details of lakhs of criminals on their fingertips. Earlier, such data was compiled separately across various state districts, and different physical records were maintained. All this made crime prevention an onerous task because police didn’t have quick access to it.

Recently, an AI-based prison monitoring system has been installed in 70 prisons across Uttar Pradesh. The AI algorithm analyses feed on hundreds of cameras installed inside prisons to detect violent acts, prison breaches, and unauthorized access in real-time to timely alert the authorities.

There is also little need now for setting up police barricades and peering inside cars to nab offenders. Tamil Nadu has built an online base of vehicles by installing an AI-based system called Tollscope, which has been linked to 30 toll plazas. If a vehicle used in a crime crosses any of the plazas, the authorities are immediately alerted. Tamil Nadu has also teamed up with Telangana police on AI-based automated systems to impose fines on two-wheeler riders without helmets. The National Crime Records Bureau has initiated deployment of a nationwide automated face recognition system to identify criminals, missing people, and unclaimed dead bodies in morgues.

Challenges in application of face recognition

In the recent past, the biometrics field has gained the utmost attention due to its reliability for recognition and easy compatibility with available technology. Due to loopholes in other identification systems, extensive research is going on in the field of biometrics. Researchers are working for the development of a more user-friendly system to pursue the requirements of security systems, which demands more accurate results for the protection of privacy and assets.

Due to a significant rise in terrorist activities, the importance of a more sophisticated security system for the identification of individuals has increased rapidly. Researchers have proposed many approaches to carry out the recognition of human faces from images and videos. Following are some of the challenges associated with the face recognition system:

Physical changes: aging; facial expression change; personal appearance (facial hair, make-up, hairstyle, glasses, disguise).

Geometry changes in acquisition: The shift in scale, in-plane rotation of the face (facing the camera), and location create geometry changes in an acquisition. The rotation, in-depth like obliquely facing camera or presentation of a profile and non-availability of the full-frontal face, also creates geometry changes in an acquisition.

Imaging changes: camera variations, lighting variation, channel characteristics (especially in broadcast or compressed images).

Reliability and cost-effectiveness are the major challenging factor for present facial recognition systems. Many researchers aim at different aspects like algorithms dealing with certain issues in face detection and face recognition itself. The natural scenes from the real environment may include several factors such as background noise, variation in lightning condition, and pose variation, which may not be present in pre-collected images.

The research also shows that the overall result has a performance issue in face recognition. Multiple samples are, therefore, quite necessary for all the techniques. These techniques can fail for special situations like ID card verification and passport verification, as there is only one image for a person.

It is difficult to detect the person from the side image or image taken from some remarkable angle. The face image with some darkness, partially brighter than remaining part of the face, blurriness, shadows, or face with spectacles, is quite difficult in the detection of any individual face.

Despite rapid advancements, face recognition is far from perfect and prone to errors. Even if a facial recognition system has an accuracy rate of 99.7%, there is always a risk of catching the wrong person because of the 0.3% error rate. Several environmental factors can affect the accuracy of a facial recognition system. Pertinently, San Francisco and Portland have banned the use of face recognition in surveillance cameras by public agencies.

The technology has turned out to be controversial, partly because of the ability to invade people’s privacy, but also because without different training data, it may work better for some types of people (mostly white nationals) than for others.

Concerns surrounding the use of face recognition

No doubt, law enforcement relies on intelligence gathering in the course of the investigation. The instinct of any police agency to gather information is irresistible and, indeed, part of its training and standard operational procedures. Most officers in their pursuit of public safety are expected to exercise a measure of reasonable restraint for the information, and particularly the quantum of information that could be gathered by facial recognition technology.

There is no harm in the use of facial recognition for mass surveillance in combination with public video cameras. However, it can also be used in a passive way that doesn’t require the knowledge, consent, or participation of the subject. The most significant danger is when this technology is used for general, suspicionless surveillance systems.

Photographs of citizens possessed by state motor vehicle agencies could easily be combined with public surveillance or other cameras in building a comprehensive system of identification and tracking. Any “batch” collection of essentially personal and individual information is a matter of concern. The delicate balance between the “reasonable expectation of privacy” and the “information as the foundation of public safety” ethic is not easy to maintain. The availability of phenomenal amounts of information in the absence of any clear plan for its use or purpose renders it liable for abuse that should concern all citizens.

Unfortunately, face recognition in authoritarian countries is rapidly becoming a routine instrument of police and government control. The practice of police scanning the faces of innocent bystanders looking for criminals runs counter to a person’s expectation of privacy in public spaces. It is not improbable that the technology can spread to the network of cameras that eventually cover the streets.

Summing up

In fact, since the invention of face recognition in the 1960s, no other single tech has sparked more fascination as well as fear of abuse than the technique of face recognition. Human rights activists fear that as a very intrusive surveillance technique, it can offer new opportunities to undermine democracy under the guise of defense.

There is a need for a balance between our reasonable expectation of privacy and the stated need for gathering information that the Constitution envisions. The new and ever-expanding wave of surveillance technology should not fail the personal privacy litmus test.

On the efficacy front, there is a need to develop an efficient real-time face recognition system that can work on a system having a training database of a single image for each individual. Most of the previous research is based on multiple images. Further, most of the researchers’ work is based on certain preconditions, which lead to many limitations. As a result of these limitations and performance issues, the practical implementation of face recognition in a real environment is quite challenging.

These issues lead us to the need for the development of real-time, more efficient face recognition applications, and a heterogeneous data matching system with human intervention. Recently, a novel approach to the hybrid face recognition technique has been introduced. This face recognition system is developed with Principle Component Analysis (PCA) and Support Vector Machine (SVM). It is checked on Oracle Research Laboratory (ORL) database as well as in the real environment. With this method based on a single image, the face recognition performance in terms of accuracy is improved. Significantly, Open Source Computer Vision, named as OpenCV, has emerged as a library of programming functions. The whole library is a cross-platform which offers real-time image processing.