How Does Facial Recognition Technology Work?

What is Facial Recognition Technology?

Facial recognition systems are a way to identify a person or confirm a person’s identity by using their face. This technology can be used to identify people in real life as well as in videos and photos. It’s a form of biometric security, which can recognize people based on their unique, inherent physical characteristics like their fingerprints.

Microsoft Defender

Stay safer online with one easy-to-use app<sup>1

1</sup>Microsoft 365 Personal or Family subscription required; app available as separate download

Learn more

How does Facial Recognition Software Work?

While there are a few biometric measurements in existence that can be used to positively identify someone, facial recognition is considered the most natural. This might be because we recognize people by looking at their faces and not their thumbprints or retinas.

You might use your face to unlock your phone or log into certain apps, and in these instances, your face identifies and recognizes you as the sole owner. Rather than trying to match your face with others in a database, it uses your face as a baseline and denies access to anyone that isn’t you. Conversely, some social media sites that feature photo sharing use facial recognition to match one tagged instance of your face in a photo to other photos within its database.

There are many facial recognition systems that exist, but they all operate in the same general way:

1. Detecting a face. A camera locates an image of a face. This face could be alone, or part of a crowd. The face can be looking straight ahead at the camera or can be turned to the side.
2. Analyzing the face. Once a face is identified, the image is taken and analyzed. Facial recognition software reads the geometry of the face and measures the distance between the eyes, the distance from the forehead to the chin, the contour of the lips, ears, and chin. It takes other measurements to identify facial landmarks that are what truly distinguishes one face from another.
3. Converting the image to data. Facial recognition software essentially converts a photo into a mathematical equation that contains all the measurements taken of the face. This equation is a numerical code that is unique to everyone, known as a faceprint.
4. Identifying a match. Once a faceprint is determined, a program can compare it against a database of other known faces until it finds a match. The software is simply looking for a numerical match to an existing faceprint, rather than comparing individual photos.

Who Uses Facial Recognition Software?

As facial recognition software is becoming more sophisticated, it’s being used for more purposes beyond unlocking a smartphone.

Law Enforcement

Facial recognition software is regularly used by law enforcement officials. When an individual is arrested and their mugshot is taken, it’s compared against other local, state, and federal databases. This can help determine an identity as well as a prior criminal record. The software also allows police to identify a person by using portable devices like smartphones and tablets to take a picture of that person and compare their photo against a faceprint database.

Facial recognition can also help to remove bias from stop and search instances by law enforcement. By making an accurate match of a person to a criminal in a database, this technology can decrease stops and searches on innocent people.

While retail security isn’t the same as law enforcement, facial recognition can help security teams recognize a shoplifter and reduce retail crime.

Airports

Waiting in line for passport control at an international border can be a tedious process. A biometric passport using facial recognition technology can improve waiting time and help tighten security, since it will be more difficult for someone to use a stolen passport. A few major airlines are even allowing passengers to board a flight simply by showing their face, rather than a ticker or a passport.

Human Trafficking

Missing persons and human trafficking can be helped thanks to facial recognition software. When a person is reported missing, their photo can be uploaded to a database and law enforcement can be alerted as soon as they are recognized by face recognition. This might happen in a retail store, airport, or other public space.

Healthcare

Facial recognition can help hospitals streamline patient care and minimize mistakes. The software can help to speed up access to patient records, registration, and can even detect pain and other emotions in patients, allowing for more personalized care. Some insurance companies allow customers to file health insurance claims which are signed with a photo instead of a signature, to minimize fraud.

Banking

Biometric passwords are becoming increasingly popular with banking institutions as there are no PINs or usernames to steal. It offers customers and banks a certain amount of peace of mind, because while passwords can be compromised, only you can log into to something with your face.

Privacy Concerns About Facial Recognition

No one likes to feel surveilled, and facial recognition software can inspire an intense, negative reaction and make people feel uncomfortable. While it’s certainly true that a massive database of faceprints can help catch a criminal, it’s also true that a government agency could track down ordinary people, too.

While sophisticated, the technology is not error-proof. There are legitimate concerns that a different camera angle or a new hair style or shaving of one’s facial hair could result in an error of misidentification.

There’s also the question of ethics when it comes to facial recognition. Is it ethical for a government to amass a giant database of citizens’ pictures without their consent? Many countries are currently trying to figure out laws to prevent privacy breeches and address legitimate ethical concerns.

How to Protect Yourself and Your Identity

If you are concerned about your faceprint being stored in a massive government database, there is technology available that might help you feel more at ease. Some American universities have developed glasses that are able to thwart facial recognition software and make the wearers undetectable. You may also consider steps like blocking your mouth or eyes when around cameras, so that the software cannot take accurate facial measurements and match your faceprint with its databases.

Additionally, it’s smart to consider how you interact with the internet. Some popular social media sites allow you to opt out of their photo tagging systems. You may also want to audit the photos you are tagged in, and un-tag yourself if you feel uncomfortable. Some photos might include identifying information that can be used to steal your identity, like a parent’s middle name or your first car.

You may also want to take stock of the Internet of Things (IoT). There are likely devices in your home or in your life that connect to the internet and use face recognition software. Video game systems are frequently culprits. If this is not something you’re comfortable with, you may want to safeguard your home network with a VPN and a firewall.