Facial Recognition
Overview
Face SDK allows performing the following comparison operations with face biometric templates:
- Verification (1:1): comparison of two face templates for belonging to the same person (comparison of two faces);
- Identification (1:N): comparison of one face template with other face templates (face search in the face database).
The recognition result is an estimate of similarity between the compared templates.
Facial Recognition Methods
Face SDK includes several recognition methods (algorithms) that differ in recognition accuracy and speed.
For detailed characteristics of the methods follow Recognition Performance section.
Method Numbering System
The first number in method name indicates the method version. The higher the version, the higher the method accuracy.
The second number indicates the approximate template creation time in milliseconds on a modern x86 processor with a frequency of ~3GHz. The slower the method, the higher its accuracy.
- x.1000 and x.300: These methods provide the highest accuracy, but they are also the slowest ones. Use case: face recognition in expert systems on large databases (more than 1 million faces).
- x.100: These methods are faster compared to the x.1000 and x.300 methods. Use case: real-time face recognition in a video stream on desktop/server platforms (x86). These methods can also be used to recognize faces in a video stream on modern mobile platforms (arm) with at least 4 cores and a core frequency of at least 1.6GHz.
- x.50 and x.30: The fastest methods. Use case: face recognition in a video stream on mobile platforms (arm).
Deprecated method numbering system (for Face SDK up to version 3.3)
The first number (6.x/7.x/8.x) indicates the method accuracy/speed:
- 7.x: These methods provide the highest accuracy, but they are also the slowest ones. Use case: face recognition in expert systems on large databases (More than 1 million faces).
- 6.x: These methods are faster compared to the 7.x methods. Use case: real-time face recognition in a video stream on desktop/server platforms (x86). These methods can also be used to recognize faces in a video stream on modern mobile platforms (arm) with at least 4 cores and a core frequency of at least 1.6GHz.
- 8.x: The fastest methods. Use case: face recognition in a video stream on mobile platforms (arm).
The second number indicates the method version. Later versions provide better quality. You can switch to them, when you update Face SDK from the corresponding earlier versions. For example, if the 6.6 method was used, we recommend that you use its new version 6.7 when upgrading Face SDK. The template size and template creation speed remain the same, but the recognition accuracy is higher.
In some cases, the latest Face SDK version may contain updates not for all groups of recognition methods. In this case you can either use the old recognition method, or compare its accuracy with the new methods from a faster group, and switch to the new faster methods if you see accuracy improvement. In addition, we provide recommendations on using the specific recognition methods when upgrading to a newer Face SDK version.
Methods by Use Cases Depending on Face SDK Version
Use case/SDK version | Expert Systems | Desktop/Server Platforms | Mobile Platforms |
SDK-2.0 | method7 | method6v2 | |
SDK-2.1 | method7 | method6v3 | |
SDK-2.2 | method7v2 | method6v4 | |
SDK-2.3 | method7v3 | method6v5 | |
SDK-2.5 | method7v6 | method6v6 | |
SDK-3.0 | method7v6 | method6v6 | method8v6 |
SDK-3.1 | method7v7 | method6v7 | method8v7 |
SDK-3.3 | method9v300 method9v1000 | method6v7 | method9v30 |
SDK-3.4 | method9v300 method9v1000 method9v300mask method9v1000mask | method6v7 | method9v30 method9v30mask |
SDK-3.11 | method9v300 method10v1000 method9v300mask method9v1000mask | method10v100 | method10v30 method9v30mask |
SDK-3.13 | method9v300 method11v1000 method9v300mask method9v1000mask | method10v100 | method10v30 method9v30mask |
SDK-3.17 | method9v300 method11v1000 method9v300mask method9v1000mask method12v1000 | method10v100 method12v100 | method10v30 method9v30mask method12v30 method12v50 |
You can speed up the methods using the AVX2 instruction set (available only on Linux x86 64-bit). If you need to use the AVX2 instruction set, move the contents of the lib/tensorflow_avx2
directory to the lib
directory. You can check the available instructions by running the command grep flags /proc/cpuinfo
.
Facial Recognition
Creating Recognizer Object
To recognize faces, create Recognizer
object by calling the FacerecService.createRecognizer
method with the recognizer configuration file as an argument.
All available configuration files are stored in the conf folder of the Face SDK distribution. The recognition method used is specified in the name of the configuration file, for example: method12v30_recognizer.xml
- method 12.30. The configuration files of the latest method version can be found by the word "latest" in their names, for example: recognizer_latest_v30.xml
.
For recognition of masked faces, you'd better use recognizer configuration files, which names include the word "mask", for example: method9v300mask_recognizer.xml
. These methods provide higher recognition accuracy for masked faces. For example, the standard 9v1000 method provides TAR=0.72 for masked faces, and the optimized 9v1000mask method provides TAR=0.85 at FAR=1E-6.
Before creating Recognizer
you can set values for the parameters below in recognizer configuration file or using Config.overrideParameter
method of FacerecService.Config
object:
num_threads
(integer) is number of threads which the model will run on (for all methods).inter_op_num_threads
(integer) is number of threads used to parallelize model nodes (only for methods 12.x). The parameter works only inexecution_mode
(integer, 1 - enabled, 0 - disabled).
Note: You can learn how to detect and recognize masked faces in our tutorial.
Extracting a Template
To extract a biometric template, use Recognizer.processing
method, pass RawSample
object as an argument. The result is Template
- interface object that stores a face template.
With the Template
object you can do the following:
- Get a method name (
Template.getMethodName
). - Save (serialize) a template in a binary stream (
Template.save
) to your disk.
The biometric template is created in two variations: short (64 bytes) and long (512 bytes). For identification you can enable the search for short templates, which will reduce the search time. In this case, the system will find a match with short templates first, and then confirm it with a match with a long template.
Face Verification (1:1)
Note! Verification is only possible if templates are extracted using Recognizers created with the same method (specifying the same configuration file).
To compare two face templates between each other, do the following:
- Extract the templates from samples using
Recognizer.processing
method or load them withRecognizer.loadTemplate
. - Use
Recognizer.verifyMatch
method, specifying twoTemplate
objects as an argument.
Verification Example (1:1)
- C++
- Python
// create face samples
const pbio::RawSample::Ptr sample1 = capturer->capture(image1);
const pbio::RawSample::Ptr sample2 = capturer->capture(image2);
// create templates
const pbio::Template::Ptr template1 = recognizer->processing(sample1);
const pbio::Template::Ptr template2 = recognizer->processing(sample2);
// verification
const pbio::Recognizer::MatchResult match = recognizer->verifyMatch(template1, template2)
# create face samples
sample1 = capturer.capture(image1)
sample2 = capturer.capture(image2)
# create templates
template1 = recognizer.processing(sample1)
template2 = recognizer.processing(sample2)
# verification
verif_results = recognizer.verify_match(template1, template2)
Face Identification (1:N)
Note! Identification is only possible if templates are extracted using Recognizers created with the same method (specifying the same configuration file).
To search for a template in the template database, follow the steps below:
Load (
Recognizer.loadTemplate
) or extract (Recognizer.processing
) the template you will compare with other templatesLoad the templates to search for matches (
Recognizer.loadTemplate
)To quickly search for short templates, create an object
TemplatesIndex
usingRecognizer.createIndex
method.With the TemplatesIndex object you can:
- Get a method name (
TemplatesIndex.getMethodName
). - Get a number of templates in the index (
TemplatesIndex.size
). - Get a specified template from the index by its number (
TemplatesIndex.at
). - Reserve memory for search (
TemplatesIndex.reserveSearchMemory
).
Note: The methods
Recognizer.createIndex
andTemplatesIndex.at
don't copy the template data but copy only pointers to the data.- Get a method name (
Search for templates in the index using the
Recognizer.search
method. Pass values for the following parameters as arguments:
- query_template: a template to be compared with the others
- templates_index: TemplatesIndex
- search_k_nearest: number of the nearest returned templates
- search acceleration:
- SEARCH_ACCELERATION_1: accelerated linear search
- NO_SEARCH_ACCELERATION: standard linear search without acceleration
Identification Example (1:N)
The example below shows the formation of a TemplateIndex
based on face detections from one image. Recognizer.search
returns search results for the requested template in the index in the amount of search_k_nearest
.
- C++
- Python
// capture the faces
const std::vector<pbio::RawSample::Ptr> samples = capturer.capture(...);
// make templates
std::vector<pbio::Template::Ptr> templates;
for(size_t i = 0; i < samples.size(); ++i)
{
const pbio::Template::Ptr templ = recognizer.processing(*samples[i]);
templates.push_back(templ);
}
// identification
const int search_k_nearest = 1;
const pbio::TemplatesIndex::Ptr index = recognizer->createIndex(templates);
const std::vector<pbio::Recognizer::SearchResult> search_results = recognizer->search(*templates[0], *index, search_k_nearest, pbio::Recognizer::SEARCH_ACCELERATION_1);
# capture the faces
samples = my_capturer.capture(...)
# make templates
templates = [my_recognizer.processing(sample) for sample in samples]
# identification
search_k_nearest = 1
index = my_recognizer.create_index(templates, 1)
search_results = my_recognizer.search([templates[0]], index, search_k_nearest, recognizer.SearchAccelerationType.SEARCH_ACCELERATION_1)
Match Result
1:1 and 1:N match results are returned in the MatchResult structure, that contains the following parameters:
- Distance between compared template vectors. The smaller the value, the higher the confidence in correct recognition.
- FAR is the False Acceptance Rate when the system mistakes images of different people as images of the same person.
- FRR is the False Rejection Rate when the system mistakes two images of the same person as images of different people.
- Score is the probability of correct recognition. Score value of float in a range [0..1] can be used as a convenient metric (along with the distance itself) for making a similarity decision.
You can set the FAR or FRR target value (FAR takes precedence) and the desired Score value in the method configuration file. Such a setting, for example, is required if, when switching to a new method, you need to leave the Score value from the previous method.
// Specify "1" to use new values, "0" to use default values
<fit_score>1</fit_score>
// Then specify new values for Score, FAR and FRR
<target_score>85</target_score>
<target_far>0.00000052</target_far>
<target_frr>0.1</target_frr>
You can also provide your own Score formula in terms of far, tar (1 - far) and distance value:
<fit_score>1</fit_score>
<use_custom_score_formula>1</use_custom_score_formula>
// This formula is an example and differs from the default one
<score_formula>(1-(log10(far)+8)/8)</score_formula>