Face Identification
OMNI includes several identification methods that differ in recognition quality and time characteristics.
More time-consuming methods provide better recognition quality (See Identification Performance for detailed characteristics of the methods).
Using Recognizer
Depending on recognition quality and time, all methods can be divided into 3 groups in connection with use cases:
- 10.30 – For real-time video identification on mobile platforms (arm).
- 10.100 – For real-time video identification on desktop platforms (x86).
- 9.300, 11.1000 – For expert systems, in which a person is searched by a photo.
Deprecated numbering system of methods used in OMNI SDK up to version 3.3
The first number (6.x/7.x/8.x) indicates the method quality/speed:
- 7.x - These methods provide the best quality, but they are also the slowest ones. Use case: face identification in server systems on large databases (More than 1 million faces).
- 6.x - These methods are faster compared to the 7.x methods. Use case: real-time face identification in a video stream on desktop/server platforms (x86). These methods can also be used to identify faces in a video stream on modern mobile platforms (arm) with at least 4 cores and a core frequency of at least 1.6GHz.
- 8.x - The fastest methods. Use case: face identification in a video stream on mobile platforms (arm).
The second number indicates the method version. Older versions were released later and, as a result, they provide better quality. You can switch to them, when you update SDK from the corresponding lower versions. For example, if the 6.6 recognizer was used, we recommend to use its new version 6.7 when upgrading SDK. The template size and template creation speed remain the same, and the identification quality becomes better.
Current numbering system of methods used in OMNI since version 3.3
The first number (starting from 9) indicates the method version. The higher the version, the better the quality.
The second number indicates the approximate template creation time in milliseconds on a modern x86 processor with a frequency of ~3GHz. The slower the method, the better its quality.
- x.1000 and x.300 - These methods provide the best quality, but they are also the slowest ones.* Use case: face identification in server systems on large databases (more than 1 million faces).
- x.100 - These methods are faster compared to the x.1000 and x.300 methods. Use case: real-time face identification in a video stream on desktop/server platforms (x86). These methods can also be used to identify faces in a video stream on modern mobile platforms (arm) with at least 4 cores and a core frequency of at least 1.6GHz.
- x.30 - The fastest methods. Use case: face identification in a video stream on mobile platforms (arm).
In some cases, the latest SDK version may contain updates not for all groups of identification methods. In this case you can either use the old identification method, or compare its quality with the new methods from a faster group, and switch to the new faster methods if you observe quality improvement. In addition, we provide recommendations on using the specific identification methods when upgrading to a newer SDK version.
Identification methods by use cases depending on OMNI version
| Use case/SDK version | Expert systems | Identification in a video stream | Mobile platforms |
| SDK-2.0 | method7 | method6v2 | |
| SDK-2.1 | method7 | method6v3 | |
| SDK-2.2 | method7v2 | method6v4 | |
| SDK-2.3 | method7v3 | method6v5 | |
| SDK-2.5 | method7v6 | method6v6 | |
| SDK-3.0 | method7v6 | method6v6 | method8v6 |
| SDK-3.1 | method7v7 | method6v7 | method8v7 |
| SDK-3.3 | method9v300 method9v1000 | method6v7 | method9v30 |
| SDK-3.4 | method9v300 method9v1000 method9v300mask method9v1000mask | method6v7 | method9v30 method9v30mask |
| SDK-3.11 | method9v300 method10v1000 method9v300mask method9v1000mask | method10v100 | method10v30 method9v30mask |
| SDK-3.13 | method9v300 method11v1000 method9v300mask method9v1000mask | method10v100 | method10v30 method9v30mask |
* - You can speed up the methods using the AVX2 instruction set (available only on Linux x86 64-bit). If you want to use the AVX2 instruction set, move the contents of the lib/tensorflow_avx2 directory to the lib directory. You can check the available instructions by running the command grep flags /proc/cpuinfo
To identify faces, create the Recognizer object by calling the FacerecService.createRecognizer method with the specified configuration file.
To create Encoder for one stream, specify the parameters processing_threads_count=true and matching_threads_count=false.
To create MatcherDB for one stream, specify the parameters processing_threads_count=false and matching_threads_count=true.
Here is the list of all configuration files and recognition methods available at the moment:
method6_recognizer.xml– method 6method6v2_recognizer.xml– method 6.2method6v3_recognizer.xml– method 6.3method6v4_recognizer.xml– method 6.4method6v5_recognizer.xml– method 6.5method6v6_recognizer.xml– method 6.6method6v7_recognizer.xml– method 6.7method7_recognizer.xml– method 7method7v2_recognizer.xml– method 7.2method7v3_recognizer.xml– method 7.3method7v6_recognizer.xml– method 7.6method7v7_recognizer.xml– method 7.7method8v6_recognizer.xml– method 8.6method8v7_recognizer.xml– method 8.7method9v30_recognizer.xml– method 9.30method9v300_recognizer.xml– method 9.300method9v1000_recognizer.xml– method 9.1000method10v30_recognizer.xml– method 10.30method10v100_recognizer.xml– method 10.100method10v1000_recognizer.xml– method 10.1000method11v1000_recognizer.xml– method 11.1000
method9v300_recognizer.xml– method 9.300method10v30_recognizer.xml– method 10.30method10v100_recognizer.xml– method 10.100method11v1000_recognizer.xml– method 11.1000
The following configuration files can be used for recognition of masked faces:
method9v30mask_recognizer.xmlmethod9v300mask_recognizer.xmlmethod9v1000mask_recognizer.xml
These methods provide better identification quality of masked faces. For example, the standard 9v1000 method provides TAR=0.72 for masked faces, and the optimized 9v1000mask method provides TAR=0.85 at FAR=1E-6
Note: You can learn how to detect and recognize masked faces in our tutorial.
You can also use configuration files that point to the latest version of the corresponding method:
recognizer_latest_v30.xmlrecognizer_latest_v100.xmlrecognizer_latest_v300.xmlrecognizer_latest_v1000.xml
Using the Recognizer object you can do the following:
- Get a method name (
Recognizer.getMethodName) - Create a template of one captured face (
Recognizer.processing) – You'll get theTemplateobject. - Match two templates (
Recognizer.verifyMatch) – You can compare only the templates created by the same method (i.e. with the same configuration file). - Load (deserialize) a template from a binary stream (
Recognizer.loadTemplate) – You can load only the templates created by the same method (i.e. with the same configuration file). - Create the
TemplatesIndexobject (Recognizer.createIndex), which is an index for quick search in large databases. - Search the
Templatein theTemplatesIndex(Recognizer.search).
1:N Identification Example
The example below shows the formation of a TemplateIndex based on face detections from one image. Recognizer.search returns search results for the requested template in the index in the amount of search_k_nearest.
- C++
- Python
// capture the faces
const std::vector<pbio::RawSample::Ptr> samples = capturer.capture(...);
// make templates
std::vector<pbio::Template::Ptr> templates;
for(size_t i = 0; i < samples.size(); ++i)
{
const pbio::Template::Ptr templ = recognizer.processing(*samples[i]);
templates.push_back(templ);
}
const int search_k_nearest = 1;
const pbio::TemplatesIndex::Ptr index = recognizer->createIndex(templates);
const std::vector<pbio::Recognizer::SearchResult> search_results = recognizer->search(*templates[0], *index, search_k_nearest, pbio::Recognizer::SEARCH_ACCELERATION_1);
# capture the faces
samples = my_capturer.capture(...)
# make templates
templates = [my_recognizer.processing(sample) for sample in samples]
search_k_nearest = 1
index = my_recognizer.create_index(templates, 1)
search_results = my_recognizer.search([templates[0]], index, search_k_nearest, recognizer.SearchAccelerationType.SEARCH_ACCELERATION_1)
With the Template object you can do the following:
- Get a method name (
Template.getMethodName). - Save (serialize) a template in a binary stream (
Template.save).
With the TemplatesIndex object you can:
- Get a method name (
TemplatesIndex.getMethodName). - Get a number of templates in the index (
TemplatesIndex.size). - Get a specified template from the index by its number (
TemplatesIndex.at). - Reserve memory for search (
TemplatesIndex.reserveSearchMemory).
Note: The methods Recognizer.createIndex and TemplatesIndex.at don't copy the template data but copy only pointers to the data.
Match Result
1:1 match result and 1:N search result contain the MatchResult structure, showing the difference between two comparing templates as distance and corresponding FAR, FRR and score values. Score value of float in a range [0..1] can be used as a convenient metric (along with the distance itself) for making a comparison decision. Since the distance threshold depends on the required indicators FAR and FRR, score metric can be adjusted to the desired target value. To achieve this you need to set the target FAR or FRR value (FAR is a priority), and the required score value in recognizers config file:
// unset, if you want to use the old score, saved in the config file
<fit_score>1</fit_score>
<target_score>85</target_score>
<target_far>0.00000052</target_far>
<target_frr>0.1</target_frr>
You can also provide your own score formula in terms of far, tar (1 - far) and distance value:
<fit_score>1</fit_score>
<use_custom_score_formula>1</use_custom_score_formula>
// this formula is an example and differs from the default one
<score_formula>(1-(log10(far)+8)/8)</score_formula>