Skip to main content
Version: 2.0.0

Benchmark Results

Accuracy Testing

Accuracy of age-estimator, gender-estimator and emotion-estimator

ServiceAccuracy
age-estimator+/- 3.95 years
gender-estimator95%
emotion-estimator80%

Accuracy of mask-estimator

Metrics:

  • Precision: the metric shows how accurate the service is and represents the number of true positive results relative to all positive results.
  • Recall: the metric indicates how completely the service covers the correct results, and represents the number of correct positive results in relation to all the results that should be positive.
  • F1 score is one of the ways to combine precision and recall metrics into an aggregate accuracy criterion. F1 score reaches its maximum at recall and precision equal to one, and is close to zero if one of the arguments is close to zero. F1 score is a harmonic mean (with a multiplier of 2, so that in the case of precision = 1 and recall = 1 get F1 = 1).

Results:

MetricValue
Precision0.9967532468
Recall0.9903225806
F1 score0.9935275081

Accuracy of face-detector-liveness-estimator

Metrics:

  • APCER: the metric shows the proportion of validation dataset attacks that were classified as real biometric presentations.
  • BPCER: the metric shows the proportion of real biometric presentations classified as attacks.

Results:

Image TypeMetricValue
real faceBPCER0.29981
photoAPCER0.04911
photo without backgroundAPCER0.12
replay attackAPCER0.01339
2D maskAPCER0.02888
3D maskAPCER0.01333
note

Average request time is equal to 0.3 s.

Accuracy of liveness-estimator module

Image TypeMetricValue
real faceBPCER0.0427664550618109
photoAPCER0.117647058823529
photo without backgroundAPCER0.160442600276625
replay attackAPCER0.246206896551724
2D maskAPCER0.0386542591267001
3D maskAPCER0.0250391236306729