Facial and body recognition

OMNI Agent detects faces and silhouettes, determines facial attributes (gender, age, emotions, etc.), generates biometric templates, performs search for detected faces in the database, and tracks the movements of faces and silhouettes within the camera's field of view or on video recordings. By default, synchronization between OMNI Agent's face database and OMNI Platform face database occurs approximately once per minute.

Facial recognition

Face detection and identification (1:N) in OMNI Agent works with detectors and recognition methods from Face SDK — a set of libraries for developing facial recognition solutions.

Identification (1:N) is the process of comparing one biometric template (face) with other templates (faces), searching and evaluating matches. When comparing face templates, matcher calculates the difference between biometric features of faces. The calculations result in a measure of correspondence between face images and the probability of belonging to one person.

Basic settings

note

All parameters listed below are configured in the config/run_params.json configuration file.

info

Basic settings can also be adjusted through the web interface. To do so, navigate to Settings in the top right corner of the OMNI Agent web interface.

Enable / disable facial recognition

Set the value for pipelines.face.enabled:(Boolean) as true (enabled) or false (disabled).

IMPORTANT

Disabling this module also stops Agent synchronization with the local database.

Facial recognition parameters

score

This parameter indicates the degree of similarity between faces, ranging from 0 (0%) to 1 (100%). A high degree of similarity means that two biometric templates belong to the same person. You can change score value in desired_score (float) field. Recommended parameter value is 0.876.

When the score is specified, faR and frR are not taken into account in recognition.

tip

Make sure the score values specified for OMNI Agent and OMNI Platform match. Otherwise, some of the activities generated from the submitted Agent processes will not be linked to the corresponding profile.

For example:

• score specified for OMNI Agent = 0.7
• score specified for OMNI Platform = 0.85

In this case, activities generated from Agent processes with a score value in the range [0.7, 0.85) will not be attached to the corresponding profile.

faR

False acceptance rate (FAR) shows the system resistance to false acceptance errors. Such an error occurs when the biometric system recognizes a new face as previously detected one. This rate is measured by the number of false-acceptance recognitions divided by the total number of recognition attempts. You can change the faR value at desired_far (float) parameter. By default, the faR value equals to 1e-5.

frR

False rejection rate (FRR). When a system fails to recognize previously detected face, false rejection occurs. The rate shows the percentage of recognition attempts with false rejection result. You can change the frR value at desired_frr (float) parameter. By default, the frR value equals to 0.

Number of identification candidates

Candidate is a profile from the database with which the identification occurred (a face from the submitted frame and a face from the database have a high degree of similarity). The number of allowable candidates can be specified in the field pipelines.face.video_worker_override_parameters.search_k: int. The default value is 3.

Enable / disable age and gender estimation

Set the value for pipelines.age_gender.enabled:(Boolean) as true (enabled) or false (disabled).

Enable / disable estimation of emotions

Set the value for pipelines.emotions.enabled:(Boolean) as true (enabled) or false (disabled).

Biometric templates

Template generation settings

When a person enters the frame, OMNI Agent builds a biometric template based on the first image of the face, which passes the image quality threshold. You can enable/disable template generation in the Settings of OMNI Agent web interface (enabled by default).

Number of template generation threads

By default, the generation of biometric templates allocates 1 core (1 thread) of the processor per 1 camera, which may be insufficient when running OMNI Agent in a multi-threaded (multi-camera) mode. To increase the number of template generation threads, open the configuration file config/run_params.json and change the value of the processing scale factor:

• pipelines.face.processing_scale_factor: float – default value is 1.5

Best shots

A best shot is the highest-quality image of a face captured within a set time interval while the person is in the frame.

Constant best shot search

By default, OMNI Agent is configured for continuous searching for the best shot throughout the entire time a person is in the camera's field of view.

For continuous searching, a temporal timeout can also be specified (default is 3000 ms). This means that during a person's presence in the frame, a search for a new best shot will be conducted every 3000 ms.

When a person leaves the frame, we obtain the highest quality image of their face for the entire track. The resulting image can be used for automatic identification against a database of wanted individuals, as well as for witness recognition, printing for security services, social media searches, etc.

Best shot search timeout

For face identification on video streams, it is important not only to ensure the accuracy but also to stay within the time allotted for identification within user scenarios (for example, for access control systems, instant identification is often necessary to avoid queues at the turnstile, while for gathering visit analytics, there may be no time limit on identification).

At the same time, accuracy and identification time are determined by the time the system can spend searching the video stream for the best shot, which will then be sent for identification.

Here, an unwritten rule applies: the more time spent on identification, the more frames of the same face can be obtained and compared, allowing for the selection of the highest quality frame to be sent for identification (best shot). And the higher the quality of the frame sent for identification, the more accurate the identification result.

Starting from version 2.7.0, the OMNI Agent web interface provides the option to configure the best shot search time. We recommend specifying the following time values depending on the use case:

Use caseBest shot search timeoutNoteACS (biometric terminal)0 ms The biometric terminal is typically configured to immediately capture a high-quality image of the face (as a person approaches the turnstile and looks at it from top to bottom). This allows for detection, searching for matches in the facial database, and opening the turnstile if the face is found in the list, all within a time frame of no more than 2 seconds.ACS (camera directed at the flow of people)3000 ms In this scenario, the camera is positioned in such a way that it detects the face several seconds before a person approaches the turnstile. This allows more time to be spent searching for the best shot and potentially obtaining a more accurate identification result.Safe city3000-5000 ms This time interval allows for enough time to be spent searching for the best shot (taking into account weather conditions, camera contamination, etc.), potentially resulting in a more accurate identification result (for example, to avoid false police/security calls).Gathering visit / movement analyticsNot limitedWhen collecting analytics, there's no time limit for identification because there's no need for an instant or rapid response to the identification result. Typically, analytics are reviewed after a certain period following the person leaving the frame (for example, the next day, once a week, etc.).

Send best shots via webhooks

To send bestshots via webhooks, enable the Send bestshots in webhook option in the Settings of the OMNI Agent web interface, or specify the value true for the enable_webhook_image2jpg_conversion field. The default value is false. The bestshot is encoded in base64 as a jpeg image.

Body detection and matching

Enable / disable body detection and matching

Setting through configuration file

Set the value for enable_body_tracking:(Boolean) as true (enabled) or false (disabled).

To exclude low-confidence matches, you can change confidence thresholds for body detection and comparison.

• Body detection confidence threshold is given in detector_confidence (float) parameter and equals to 0.9 by default with a range of values from 0 to 1.

• Body comparison confidence threshold is given in cos_reident_confidence (float) parameter and equals to 0.6 by default with a range of values from -1 to 1.

Setting via dashboard

You can enable or disable body detection in Settings of OMNI Agent dashboard.