7. Administration
7.1 Sending Emails to an SMTP Server on Deployed OMNI Platform
Stop the deployed OMNI Platform:
$ ./cli.sh platform uninstall
Make changes to
platform-email-secret
in the ./cfg/platform.secrets.json file.Recreate secrets:
$ ./cli.sh platform install-secrets
Deploy OMNI Platform:
$ ./cli.sh platform install
7.2 Loading a Face Database
To upload the database of face images, use the script from the utils/upload_script folder of OMNI Platform distribution kit. The script accepts a file that contains paths to dataset images, and sends requests to OMNI Platform to create profiles from these images.
The result is a list of profiles in OMNI Platform and a list of files in the script_files folder:
result.txt: A file with a number that indicates the number of images processed the last time the script was run. This number also includes images processed with errors. The number is updated while the script is running.
errors.txt: A file with image upload errors written in the format:
<path to image>
:<error>
trace_errors.txt: A file with a detailed description of the image upload error in the format:
<path to the image>
:<error> <traceback>
. You can use this file by yourself when you try to fix an error, or attach it to your email to the technical support team.
System requirements
- OS: Ubuntu 20.04
- Python: v3.8
- pip: v22.0.4
All subsequent steps are performed from the upload_script folder.
Preparation
Make sure you have a file that contains the paths to the face images. The path to each image must start on a new line without indentation.
For example:
/home/stranger/Downloads/1.jpeg
/home/stranger/Downloads/2.jpeg
/home/stranger/Downloads/3.jpegIf you don't have this file, generate it using a command that will recursively find all jpg or jpeg files in the dataset and display the absolute paths to them in the file:
find <absolute path to the dataset with images> -type f -name "*.jpg" -o -name "*.jpeg" > image_paths.txt
-name <regular expression>
specifies the image format, or rather the search condition.-o
specifies a logical OR relationship between two conditions.
Generate a virtual environment to run the image upload script:
python3 -m venv generate_persons_env
Connect to the generated environment:
source generate_persons_env/bin/activate
Install the necessary dependencies in the virtual environment:
pip3 install -r requirements.txt
Operation
Launching a scipt
The script is launched only from the virtual environment created at the preparation stage by executing the generate_persosn.py file:
python generate_persons.py
You can start creating profiles from a previously generated list of image paths using the following command:
python generate_persons.py -p ./image_paths.txt -H <platform url> -t <platform token> -P <profile group id>
Required arguments:
-p --paths
Path to a file with paths to face images.
-H --host
URL of OMNI Platform instance where images are to be uploaded. Make sure the URL is spelled without a slash in the end.
-t --token
User access token/authorization key that grants access to upload images to OMNI Platform. The token is available on the home page of OMNI Platform web interface after authorization.
Optional arguments:
-T --throughput
A parameter determines the maximum number of profile creation requests that the server executes during script running. It is recommended not to change this setting without calculating OMNI Platform performance on your server machine configuration.
-l --limit <limit>
The parameter limits the number of uploaded images if the dataset contains more images than required for uploading.
P --profile_group
uuid4 of profile group where all profiles created from uploaded images should be added to. Adding to a profile group is required to configure notifications for created profiles. uuid4 can be obtained from OMNI Platform web interface by clicking on the five digits in the upper left corner of the Watchlist card.
-c --clear-prev
The parameter ignores the previous results of script running and starts uploading images again by clearing the result.txt file. Note that in this case the errors.txt and trace_errors.txt files will also be cleared. To keep the logs, copy these files to another location.
You can get a full description of the script parameters by running the following command:
python3 generate_persons.py -h
The script is running successfully when new profiles from dataset images appear in OMNI Platform as well as console messages are obtained in the format: Handled <number of processed images> images
Stopping a script
You can stop the script with Ctrl+C. All previous processing will be saved and, if necessary, you will be able to resume the uploading.
Image Upload Errors
A message about the failed image uploading will be printed to the console, while running the script. The message itself will be saved to the errors.txt file, more detailed error information will be written to the trace_errors.txt file.
- sort: cannot read: ./local_im: No such file or directory
Incorrect path to the file that contains absolute paths to dataset images. Check that the file is located at the given path.
- <class 'aiohttp.client_exceptions.ClientResponseError'>|404, message='Not Found', url=URL('http://sadasdasd/api/v2/')
<class 'aiohttp.client_exceptions.ClientConnectorError'>|Cannot connect to host asdasd:80 ssl:default [Name or service not known]
<class 'aiohttp.client_exceptions.InvalidURL'>|http:///asdasd/api/v2/
Incorrect URL of OMNI Platform instance. Check that OMNI Platform is indeed located at the specified URL.
- <class 'Exception'>|[{'message': 'User has no access or id is incorrect'}]
The user access token is incorrect. Check that you copied the token correctly.
- <class 'Exception'>|[{'message': 'One or several profiles_groups does not exist', 'code': '0x573bkd35'}]
<class 'Exception'>|[{'message': '“%(value)s” is not a valid UUID.'}]
No profile group found by the passed id. Check that you copied the profile group id correctly.
- <class 'Exception'>|[{'message': 'No faces found', 'code': '0x95bg42fd'}]
<class 'Exception'>|[{'message': 'Low quality photo', 'code': '0x86bd49dh'}]
No faces found in the image, or poor image quality. In this case, the profile for this image will not be created in OMNI Platform.
7.3 Database Backup and Restore
Not supported for external databases.
Commands are executed from the setup directory.
Database backup
To back up the database, run the command below:
$ ./cli.sh platform db-backup - <postgres_user> <dump_path>
The <root_postgres_user>
name can be found in the ./cfg/platform.secrets.json file in postgres-root-credentials.user
Database restore
Restoring data from a backup must be performed on the same version of the platform from which the backup was created.
To restore the database, run the following commands:
Stop image-api:
$ ./cli.sh image-api uninstall
Request a list of services:
$ kubectl get deployment
Stop all services except platform-postgres-dep:
$ kubectl scale deployment <SERVICE_NAME_1> <SERVICE_NAME_2> --replicas=0
Restore data from a backup:
$ ./cli.sh platform db-restore - <dump_path>
Restart stopped services:
$ kubectl scale deployment <SERVICE_NAME> --replicas <COUNT>
or redeploy OMNI Platform
$ ./cli.sh platform install
Run image-api:
$ ./cli.sh image-api install
7.4 Registration of New Users
Registration of new users via web interface
- Follow the link http://$DOMAIN/sign-up and enter email and password of the new user.
- Sign in to the specified email and follow the confirmation link in the incoming letter.
If the SMTP server is not configured, you can confirm registration through the administrator dashboard:
- Follow the link http://$DOMAIN/storage/admin and enter administrator login and password (
PLATFORM_ADMIN_EMAIL
andPLATFORM_ADMIN_PASSWORD
) to sign in. - Then click on the Users tab, open the new user’s page in users’ list and put Active status in the Permissions section.
- After that click Save and go to the Sign-In page by clicking VIEW SITE on top of the screen.
- Registration is confirmed! At http://$DOMAIN/sign-in enter email and password to sign in to the workspace.
Registration of new users using cURL-requests
- Use cURL-template below to send API-request for registration of the new user:
curl --location 'http://$DOMAIN' --header 'Content-Type: application/json' --data-raw '{"query":"mutation ($userInfo: RegistrationInput!) {registration(userInfo: $userInfo) {ok}}","variables":{"userInfo":{"password":"password","confirmPassword":"password","email":"email@email.com"}}}'
Fill in the following fields:
http://$DOMAIN
- URL of the deployed Platform server"password":"password"
- Enter password"confirmPassword":"password"
- Confirm password"email":"email@email.com"
- Enter email
API should return "ok": true.
- Confirm registration via email, or administrator dashboard (if SMTP-server is not configured).
- Registration is confirmed! At http://$DOMAIN/sign-in enter email and password to sign in to the workspace.
The system automatically creates a separate workspace for each new user. Connecting multiple users to the same workspace is not supported.
7.5 Updating a License
Preconditions
Installed OMNI Platform with an activated license.
Instructions for updating the license
Copy the license file face_sdk.lic to the setup folder. If the file name is different, rename it to face_sdk.lic.
If the license key has been updated (a new license has been issued, not an existing one has been renewed), update the license key in the files ./cfg/platform.secrets.json and ./cfg/license-server.settings.cfg .
Change to the setup directory and recreate the secrets:
./cli.sh platform install-secrets`
Expected output:
secret "rabbit-secret" deleted
secret/rabbit-secret created
...
secret "face-sdk" deleted
secret/face-sdk created
name, server or password is empty, "docker-regcreds" secret creation skipped
secret "postgres-creds-pack" deleted
secret/postgres-creds-pack createdActivate the license:
./cli.sh license-server activate`
Expected output:
[2023-06-20 13:58:49+03:00] INF Activating license key...
[2023-06-20 13:58:50+03:00] INF License activated successfully!Restart the service:
kubectl delete $(kubectl get pod -o name)
Expected output:
pod "activity-matcher-dep-5ffdb45cbf-kkmd7" deleted
...
pod "securos-integration-service-dep-6669d88d8b-mdwff" deleted
pod "verify-matcher-dep-5695b46bd5-nvlr7" deletedRun the command:
watch -n 0.1 kubectl get pod
and wait until all pods have transitioned to the Running status.
Expected output:
NAME READY STATUS RESTARTS AGE
activity-matcher-dep-6fdc8bfbd5-pl8ql 1/1 Running 2 (24h ago) 24h
age-estimator-dep-544cdfd7c-4khhh 1/1 Running 0 24h
...
securos-integration-service-dep-77f98b497d-66q 1/1 Running 0 24h
verify-matcher-dep-85ddfdfd4f-7t7br 1/1 Running 0 24hEnd the command with the Ctrl+C.
Check that the license term has been updated:
./cli.sh license-server status-license
As a result, the license key and the number of days left to expiration should be displayed. Expected output:
Activation Status: OK
License key: YOUR-LICENSE-KEY
Days Left To Expiration: 10
7.6 Increasing the Speed of Extracting a Biometric Template
Changing this configuration only affects the search(sourceImage) API-request.
To increase the speed of extracting a biometric template, open the configuration file ./cfg/image-api.values.yaml, go to face-detector-template-extractor.configs.recognizer.params, add field num_threads: <>
and enter the required number of threads in the field.
For maximum performance, the number of threads must be selected relative to the configuration of OMNI Platform server. The number of threads should not exceed the number of processor cores!