Test results
Load testing
Load testing helps evaluate the quality and speed of OMNI Platform operation under a certain load at a given time interval. A test image used is 240 KB (512х512 px) png image.
Test machine specification
| CPU | RAM | SSD |
| 128 GB DDR4 2400 ECC | Samsung EVO 870 500 GB |
Metrics
| Metric Name | Description |
| Users | Number of parallel requests |
| Num threads | Number of CPU threads in use |
| RPS | Number of requests processed per second |
| (50%, 60%, ...) 50th, 60th, ... percentile | % of requests are completed in no more than this time, in ms |
| Max | Maximum time to process 1 request, in ms |
| Avg | Average time to process 1 request, in ms |
Basic load test
Test conditions
- Face recognition method: 12v1000
- ipvs: enabled
- Database size: two mln profiles
- Scalability: load testing of OMNI Platform 1.18.0 uses the following scaling settings:
- image-api-age-estimator-dep: 8 instances
- image-api-emotion-estimator-dep: 8 instances
- image-api-face-detector-liveness-estimator-dep: 8 instances
- image-api-mask-estimator-dep: 8 instances
- image-api-face-detector-template-extractor-dep: 16 instances
- platform-processing-dep: 16 instances
- other services: 1 instance for each service
Test results
| Request* | Users | Num_ threads | RPS | Avg | Min | Max |
| create_profile (create a profile from a face image) | 1 | 1 | 0.9 | 1073 | 994 | 1273 |
| 16 | 1 | 11 | 1395 | 1193 | 3271 | |
| detect (detect a face and estimate facial attributes from a face image) | 1 | 1 | 1 | 973 | 886 | 1189 |
| 16 | 1 | 12 | 1316 | 1094 | 3029 | |
| profiles (get a list of created profiles) | 1 | 1 | 4.9 | 205 | 157 | 383 |
| 230 | 1 | 215 | 1066 | 343 | 4587 | |
| search (search a face in the database by image) | 1 | 1 | 0.96 | 1045 | 943 | 2140 |
| 16 | 1 | 12.5 | 1278 | 1015 | 3807 | |
| search_by_ sample_id (search a face in the database by sample ID) | 1 | 1 | 6.9 | 145 | 116 | 2321 |
| 70 | 1 | 33 | 2114 | 113 | 5277 | |
| verify (compare a face image with a sample to determine if they belong to the same person) | 1 | 1 | 1 | 988 | 907 | 1148 |
| 16 | 1 | 12 | 1330 | 1116 | 3350 | |
| verify_by_ sample_id (compare two samples to determine if they belong to the same person) | 1 | 1 | 16 | 62 | 46 | 283 |
| 260 | 1 | 380 | 667 | 223 | 1120 |
CONCLUSION
The most time-consuming operation "Search" takes
- 0.5 seconds for single requests
- 1 second under a load of 16 parallel requests
/* Detailed information on the API requests specified in the table can be found in API Reference.
Load test for receiving events/activities from cameras
| Check type | Avg (ms) | Min (ms) | Max (ms) | RPS |
| Receiving events | 5001 | 156 | 26882 | 24.8 |
| Receiving activities | 1372 | 349 | 6169 | 66.5 |
CONCLUSIONS
During the test, OMNI Platform successfully processes the activity reception load at a rate of 1800 activities/sec (300 cameras are connected to OMNI Platform, each of which sends activities at a rate of 6 activities/sec).
The event reception load is processed at a rate of 200 events/sec (200 cameras are connected to OMNI Platform, each of which sends events at a rate of 1 event/sec).