Skip to main content
Version: 3.28 (latest)

Quality assessment

Facial Image Quality Assessment Processing Block​

Facial Image Quality Assessment evaluates how well a facial image complies with quality standards such as ISO/IEC 19794-5:2011 and the ICAO Guidelines for Passport Photographs.

This procedure applies a wide range of quality checks. However, for modern facial recognition methods, not all of these checks are equally critical to recognition accuracy.

If your goal is to perform face quality control specifically to minimize facial recognition errors or support Liveness Detection, refer to Quality control.

Quality Assessment performance​

Modification Version Face SDK version Detection time CPU (ms)* Detection time GPU (ms)** The worst samples by total_score, discarded, (%)FNMRFPR
assessment13.16595710.00971.98E-07
50.00872.00E-07
100.00812.11E-07
23.24626110.00971.98E-07
50.00872.00E-07
100.00812.11E-07
* - CPU Intel Xeon E5-2683 v4 (single-core)
** - GPU (NVIDIA GTX 10xx series)

Creation and configuration of assessment block​

The "assessment" mode is the first implemented option within the Quality Assessment module, designed primarily for checking whether facial images comply with standards such as ISO/IEC 19794-5:2011 or the ICAO Guidelines for Passport Photographs. The results of individual checks are aggregated into a single metric, total_score, which reflects how well the image meets the standard.

Currently, it is recommended to use the second version of the assessment mode. Note: for accurate evaluation, the image must contain only one face looking directly at the camera. Otherwise, the final score may be underestimated. If multiple faces are captured in the same image, each will be processed independently.

Parameters evaluating the 1st version of "assessment" modification​

Click to display the list of parameters being evaluated
  • total_score — a numeric value that indicates the overall image quality score from 0 to 1.
  • is_background_uniform — a boolean value, indicates the uniformity of the background.
  • background_uniformity_score — numeric value, indicates the background uniformity score in points from 0 to 1.
  • is_dynamic_range_acceptable — a logical value, indicates that the dynamic range of the image intensity in the facial area is greater or less than the value of 1.28.
  • dynamic_range_score — numeric value, a dynamic intensity range score from 0 to 2.55.
  • is_eyes_distance_acceptable — a logical value, indicates the acceptable/unacceptable distance between eyes.
  • eyes_distance — numeric value of the distance between eyes in pixels.
  • is_evenly_illuminated — boolean value, denotes uniformity of illumination in the image.
  • illumination_score — numeric value, indicates the uniformity of illumination in points from 0 to 1.
  • no_flare — a logical value, indicates the presence or absence of flare in the image.
  • is_left_eye_opened — logical value, indicates the position of the left eye (open or closed).
  • left_eye_openness_score — numeric value, indicates the degree of eye openness in points from 0 to 1.
  • is_right_eye_opened — logical value, indicates the position of the right eye (open or closed).
  • right_eye_openness_score — numeric value, indicates the degree of eye openness in scores from 0 to 1.
  • is_neutral_emotion — logical value, denotes the presence or absence of a neutral facial expression.
  • neutral_emotion_score — numerical value, scores the degree of neutral emotion in points from 0 to 1.
  • is_not_noisy — logical value, denotes the presence or absence of noise in the image.
  • noise_score — numerical value, evaluation of image noise in scores from 0 to 1.
  • is_sharp — logical value, denotes sharpness of the image.
  • sharpness_score — numeric value, denotes the sharpness score in points from 0 to 1.
  • is_margins_acceptable — a logical value, denotes acceptable/unacceptable indents.
  • margin_inner_deviation — numeric value of internal deviation in pixels.
  • margin_outer_deviation — numeric value of outer deviation in pixels.
  • is_rotation_acceptable — boolean value, indicates acceptable/unacceptable head rotation.
  • max_rotation_deviation — numeric value, maximum degree of deviation for three (yaw, pitch, roll) head rotation angles.
  • not_masked — boolean value, indicates the presence or absence of a mask on the face.
  • not_masked_score — numeric value, indicates the degree of confidence in the absence of a mask on the face from 0 to 1.
  • has_watermark — boolean value, indicates the presence or absence of a watermark on the image.
  • watermark_score — numeric value, indicates the degree of confidence in the presence of a watermark on the image from 0 to 1.

Parameters evaluating the 2nd version of "assessment" modification​

Click to display the list of parameters being evaluated
  • total_score — a numeric value that indicates the overall image quality score from 0 to 1.
  • is_background_uniform — a boolean value, indicates the uniformity of the background.
  • background_uniformity_score — numeric value, indicates the background uniformity score in points from 0 to 1.
  • is_dynamic_range_acceptable — a logical value, indicates that the dynamic range of the image intensity in the facial area is greater or less than the value of 1.28.
  • dynamic_range_score — numeric value, a dynamic intensity range score in a score from 0 to 2.55.
  • is_eyes_distance_acceptable — a logical value, indicates the acceptable/unacceptable distance between eyes.
  • eyes_distance — numeric value of the distance between eyes in pixels.
  • is_evenly_illuminated — boolean value, denotes uniformity of illumination in the image.
  • illumination_score — numeric value, indicates the uniformity of illumination in points from 0 to 1.
  • no_flare — a logical value, indicates the presence or absence of flare in the image.
  • is_left_eye_opened — logical value, indicates the position of the left eye (open or closed).
  • left_eye_openness_score — numeric value, indicates the degree of eye openness in points from 0 to 1.
  • is_right_eye_opened — logical value, indicates the position of the right eye (open or closed).
  • right_eye_openness_score — numeric value, indicates the degree of eye openness in scores from 0 to 1.
  • is_neutral_emotion — logical value, denotes the presence or absence of a neutral facial expression.
  • neutral_emotion_score — numerical value, scores the degree of neutral emotion in points from 0 to 1.
  • is_not_noisy — logical value, denotes the presence or absence of noise in the image.
  • noise_score — numerical value, evaluation of image noise in scores from 0 to 1.
  • is_sharp — logical value, denotes sharpness of the image.
  • sharpness_score — numeric value, denotes the sharpness score in points from 0 to 1.
  • is_margins_acceptable — a logical value, denotes acceptable/unacceptable indents.
  • margin_inner_deviation — numeric value of internal deviation in pixels.
  • margin_outer_deviation — numeric value of outer deviation in pixels.
  • is_rotation_acceptable — boolean value, indicates acceptable/unacceptable head rotation.
  • max_rotation_deviation — numeric value, maximum degree of deviation for three (yaw, pitch, roll) head rotation angles.
  • not_masked — boolean value, indicates the presence or absence of a mask on the face.
  • not_masked_score — numeric value, indicates the degree of confidence in the absence of a mask on the face from 0 to 1.
  • has_watermark — boolean value, indicates the presence or absence of a watermark on the image.
  • watermark_score — numeric value, indicates the degree of confidence in the presence of a watermark on the image from 0 to 1.
  • face_width_pixels - numeric value, indicates the width of the face in pixels
  • face_height_pixels - numeric value, indicates the height of the face in pixels
  • face_size_on_image - numeric value, indicates the size of the face on the image from 0 to 1 (ICAO parameter)
  • is_face_size_acceptable - logical value, indicates the acceptable size of the face on the image (70-80% of the image area) (ICAO parameter)
  • is_image_gray - logical value, indicates whether the image is gray (ICAO parameter)
  • min_image_dpi - numeric value, indicates the number of pixels per inch (ICAO parameter)
  • is_high_resolution - logical value, indicates whether the image is a high-quality image (ICAO parameter)
  • image_width_to_height_ratio - numeric value, indicates the ratio of the width to the height of the image
  • red_eyes_right_score - numeric value, right eye redness score (ICAO parameter)
  • red_eyes_left_score - numeric value, left eye redness score (ICAO parameter)
  • is_red_eyes - logical value, indicates the presence of red eyes (ICAO parameter)
  • not_has_glasses - logical value, indicates the absence of glasses (ICAO parameter)
  • glasses_score - numeric value, indicates the presence of glasses score from 0 to 1 (ICAO parameter)
  • mouth_openness_score - numeric value, mouth openness score from to 1 (ICAO parameter)
  • is_mouth_openness - logical value, indicates whether the mouth is open (ICAO parameter)
Key points

For the better results when assessing image quality using the 2nd version of "assessment" modification, use FACE_FITTER with tddfa or tddfa_faster modification.

Checks

The 2nd version of "assessment" modification lets you disable some checks. Disabled checks will not affect the final result.

Disabled checks of the 2nd version of "assessment" modification​

Click to display the list of disabled parameters. All options are enabled by default
  • enable_check_sharpness - boolean, sharpness check
  • enable_check_illumination - boolean, illumination uniformity check
  • enable_check_flare - boolean, flare check
  • enable_check_noise - boolean, noise check
  • enable_check_dynamic_range - boolean, intensity dynamic range check
  • enable_check_watermark - boolean, watermark check
  • enable_check_background - boolean, background uniformity check
  • enable_check_eye_openness - boolean, eye openness check
  • enable_check_mask - boolean, mask check
  • enable_check_neutral_emotion - boolean, emotion check
  • enable_check_eye_distance - boolean, check the distance between the eyes
  • enable_check_margin - boolean, check the margins
  • enable_check_rotation - boolean, check the head rotation
  • enable_check_resolution - boolean, check the image resolution
  • enable_check_gray_scale - boolean, check if the image is gray
  • enable_check_face_size - boolean, check the face size
  • enable_check_red_eye - boolean, check if the eyes are red
  • enable_check_glasses - boolean, check if the glasses are on
  • enable_check_mouth_openness - boolean, check if the mouth is open

Specification of the Quality Assessment input Context​

Processing Block expects an input Context containing a binary image and an array of objects obtained after the work of the Face Detector and Face Fitter Processing Blocks:

Click to expand the Context input container specification
```json
{
    "image" : {
    "format": "NDARRAY",
        "blob": "data pointer",
        "dtype": "uint8_t",
        "shape": [height, width, channels]
    },
    "objects": [{
        "id": {"type": "long", "minimum": 0},
        "class": "face",
        "confidence": {"double",  "minimum": 0,  "maximum": 1},
        "bbox": [x1, y2, x2, y2],
        "keypoints": {
            "left_eye_brow_left":   {"proj" : [x, y]},
            "left_eye_brow_up":     {"proj" : [x, y]},
            "left_eye_brow_right":  {"proj" : [x, y]},
            "right_eye_brow_left":  {"proj" : [x, y]},
            "right_eye_brow_up":    {"proj" : [x, y]},
            "right_eye_brow_right": {"proj" : [x, y]},
            "left_eye_left":        {"proj" : [x, y]},
            "left_eye":             {"proj" : [x, y]},
            "left_eye_right":       {"proj" : [x, y]},
            "right_eye_left":       {"proj" : [x, y]},
            "right_eye":            {"proj" : [x, y]},
            "right_eye_right":      {"proj" : [x, y]},
            "left_ear_bottom":      {"proj" : [x, y]},
            "nose_left":            {"proj" : [x, y]},
            "nose":                 {"proj" : [x, y]},
            "nose_right":           {"proj" : [x, y]},
            "right_ear_bottom":     {"proj" : [x, y]},
            "mouth_left":           {"proj" : [x, y]},
            "mouth":                {"proj" : [x, y]},
            "mouth_right":          {"proj" : [x, y]},
            "chin":                 {"proj" : [x, y]},
            "points":               {"proj" : [x, y]}
        }
    }]
}
```

Output Context Specification​

After calling the quality assessment processing block, attributes corresponding to this block will be added to each object from the “objects” array. Context output container specification:

{
    "quality": {
        "total_score": {"type": "double", "minimum": 0, "maximum": 1},
        "is_sharp": {"type": "boolean"},
        "sharpness_score": {"type": "double", "minimum": 0, "maximum": 1},
        "is_evenly_illuminated": {"type": "boolean"},
        "illumination_score": {"type": "double", "minimum": 0, "maximum": 1},
        "no_flare": {"type": "boolean"},
        "is_left_eye_opened": {"type": "boolean"},
        "left_eye_openness_score": {"type": "double", "minimum": 0, "maximum": 1},
        "is_right_eye_opened": {"type": "boolean"},
        "right_eye_openness_score": {"type": "double", "minimum": 0, "maximum": 1},
        "is_rotation_acceptable": {"type": "boolean"},
        "max_rotation_deviation": {"type": "long"},
        "not_masked": {"type": "boolean"},
        "not_masked_score": {"type": "double", "minimum": 0, "maximum": 1},
        "is_neutral_emotion": {"type": "boolean"},
        "neutral_emotion_score": {"type": "double", "minimum": 0, "maximum": 1},
        "is_eyes_distance_acceptable": {"type": "boolean"},
        "eyes_distance": {"type": "long", "minimum": 0},
        "is_margins_acceptable": {"type": "boolean"},
        "margin_outer_deviation": {"type": "long", "minimum": 0},
        "margin_inner_deviation": {"type": "long", "minimum": 0},
        "is_not_noisy": {"type": "boolean"},
        "noise_score": {"type": "double", "minimum": 0, "maximum": 1},
        "watermark_score": {"type": "long", "minimum": 0},
        "has_watermark": {"type": "boolean"},
        "dynamic_range_score": {"type": "double", "minimum": 0},
        "is_dynamic_range_acceptable": {"type": "boolean"},
        "is_background_uniform": {"type": "boolean"},
        "background_uniformity_score": {"type": "double", "minimum": 0, "maximum": 1},
        "face_width_pixels": { "type": "long" },
        "face_height_pixels": { "type": "long" },
        "face_size_on_image": { "type": "double", "minimum": 0, "maximum": 1 },
        "is_face_size_acceptable": { "type": "boolean" },
        "is_image_gray": { "type": "boolean" },
        "min_image_dpi": { "type": "long" },
        "is_high_resolution": { "type": "boolean" },
        "image_width_to_height_ratio": { "type": "double" },
        "red_eyes_right_score": { "type": "double" },
        "red_eyes_left_score": { "type": "double" },
        "is_red_eyes": { "type": "boolean" },
        "glasses_score": { "type": "double", "minimum": 0, "maximum": 1 },
        "not_has_glasses": { "type": "boolean" },
        "mouth_openness_score": { "type": "double", "minimum": 0, "maximum": 1 },
        "is_mouth_openness": { "type": "boolean" }
    }
}

Example of working with the Quality Assessment processing block​

  1. Create a Context configuration container and specify the values "unit_type", "modification", "version" of the block you are interested in. To create a processing block, follow the Working with Processing Block page.

  2. Pass the container-context obtained after the Face Detector and Face Fitter processing blocks.

  3. Call the evaluation processing block.

  4. Get the result of the processing block.

```cpp

auto configCtx = service->createContext(); configCtx["unit_type"] = "QUALITY_ASSESSMENT_ESTIMATOR"; pbio::ProcessingBlock blockQuality = service->createProcessingBlock(configCtx);

//------------------ // creation of face detection processing blocks, and Context container with binary image //------------------

faceDetector(ioData); faceFitter(ioData); blockQuality(ioData);

double total_score = ioData["objects"][0]["quality"]["total_score"].getDouble();

</TabItem>

<TabItem value="py">

```py
configCtx = {"unit_type": "QUALITY_ASSESSMENT_ESTIMATOR"}
blockQuality = service.create_processing_block(configCtx)

#------------------
# creation of face detection processing blocks, and Context container with binary image
#------------------

faceDetector(ioData)
faceFitter(ioData)
blockQuality(ioData)

total_score = ioData["objects"][0]["quality"]["total_score"].get_value()