Face SDK Components
Face Detector
Face Detector is a component for detecting faces in images. The result of the component's work is a list of detected faces.
For each detected face, the following attributes are available:
- Bounding Box - position and size of the face in the original image
- Face Landmarks - 2D coordinates of the anthropometric points of the face
- Face Mesh - 3D coordinates of 470 anthropometric points of the face
- Iris Landmarks - coordinates of 40 points of the eyes (pupils and eyelids)
- Head Rotation Angles
Note: For face recognition on videos or time-ordered image sequences, it is recommended to use the Video Engine component.
Attributes Estimators
Face SDK provides a set of tools for estimating images of faces received from the Face Detector component.
Gender-Age Estimator is used for estimating the gender and age of people by using their face images.
| Attribute | Accuracy |
|---|---|
| Gender | 95% |
| Age | Average error +/-4 years |
Emotions Estimator is used for estimating the prevailing emotional state of a person:
- Happy
- Surprised
- Neutral
- Angry
| Attribute | Accuracy |
|---|---|
| Emotions | 80% |
Quality Estimator provides an appraisal of face image quality as a single number or a set of separated appraisals of quality parameters:
- flare level
- uniformity of illumination
- noise level
- sharpness level
Mask Estimator determines the presence/absence of a mask on the face.
| Attribute | Accuracy |
|---|---|
| No mask (face without a mask) | 99% |
| Has mask (face with a mask) | 97% |
Eyes Openness Estimator is used for estimating the eyes’ state on the face image. This component provides the verdict “open” or “closed” for the right and left eye.
Face Recognition
Face SDK provides components and algorithms to recognize and compare faces. This functionality is based on the operations with a biometric face template.
A biometric face template is a unique set of biometric features extracted from a face image. Templates are used to compare two face images and to determine a degree of similarity
A biometric face template has the following key characteristics:
- It does not contain personal data
- It cannot be used to restore face image
- It can be serialized and saved to file, database, or sent over a network
- It can be indexed. That helps accelerate face template matching process by using a special index for face template batch.
Encoder extracts biometric face template from a face image received from Face Detector.
Face SDK has several algorithms with different characteristics of speed and accuracy for all possible use cases - from low-powered embedded devices to expert face recognition systems.
Extracting a biometric template is one of the most computation-heavy operations, so Face SDK provides the ability to use a GPU accelerator to increase performance.
Matcher allows performing the following comparison operations with templates created by Encoder:
- verification 1:1 - comparing of two biometric templates (faces) between each other, estimating of coincidence
- identification 1:N - omparing of one biometric template (face) with other templates (faces), searching and estimating of coincidence
When comparing face templates, Matcher calculates the difference between biometric features of faces. The calculations result in a measure of the correspondence for the images of faces and the probability of belonging to one person.
Templates, extracted using different algorithms, have different properties and cannot be compared with each other.
Face SDK accuracy by NIST (28-10-2021)
| NIST Face Recognition Vendor Test (FRVT) 1:1 | Score |
|---|---|
| VISA, True Acceptance Rate (@FAR 1E-6) | 99.62% |
| MUGSHOT, True Acceptance Rate (@FAR 1E-5) | 99.72% |
| VISA BORDER, True Acceptance Rate (@FAR: 1E-6) | 99.54% |
Liveness Estimators
The Liveness components are used to assess whether the face detected in the image or in video is real or fake. These components protect against malicious actions (spoofing attacks) using a photo or video image instead of a real face.
Active Liveness Estimator analyzes certain human actions according to the check script, for example: “blink”, “smile”, “turn your head” in order to distinguish a real live face from video or photo image
2D / RGB Liveness Estimator assesses the liveness of a face in an RGB image. To perform the check, the appearance of the face in the field of view of the camera is sufficient.
3D / Depth Liveness Estimator protects against attempts to use an image instead of a real face by analyzing the face surface using a depth map obtained from a 3D (RGBD) sensor.
IR Liveness Estimator determines the reality of a human face based on an image taken from an infrared camera, combined with a color image.
Video Engine
Video Engine is used for real-time processing of video streams. This component solves the following tasks:
- face detection and tracking
- face recognition (optional)
- Liveness checking (optional)
- determination of age, gender, and emotions (optional)
Video Engine works in a multi-stream mode. Each stream is a sequence of images (frames) obtained from one source (for example, a camera or video).
All streams are processed by Video Engine at the same time. Streams, frames, and detected faces in the frame are assigned their own identifiers. During face tracking, a track of this face is formed on the sequence of stream images, which is also indicated by its own ID.
The set of identifiers allows you to accurately trace the generated events for each stream. To handle the events, Video Engine implements a callback interface that provides event data and allows you to define application behavior.
Licensing
Face SDK supports all major platforms and is available on flexible license terms. Platform licensing or fine-grained licensing of individual features is possible upon request.