Installation instructions
Deployment preparation
Downloading
Download and unpack the BAF distribution kit onto the machine where you plan to install it. To do this, use the command below:
$ curl --output baf.zip distribution link
In the curl request, specify a link to the BAF distribution kit (zip file). A link to the folder that contains the distribution kit and pdf documentation will be sent to you in the email.
Next, move the license file face_sdk.lic (the file is attached to the email) to the setup folder.
Ensure the license file is named face_sdk.lic. Using any other name will result in errors during secrets generation.
Contents of the distribution kit:
- ./cli.sh: entry point to run the commands
- ./cfg: folder with configuration files
Further commands are to be executed in the system console from setup directory.
Uploading of images
First, upload the product images from the archive to the local registry:
$ ./cli.sh generic load-images
Then, upload the infrastructure images from the archive to the local registry:
$ ./cli.sh smc load-images
Configuration
Basic configuration
Enter environment variables
Globally, environment variables can be divided into three categories:
- Environment variables of system elements not related to Helm charts;
- Public environment variables of the module chart;
- Secret environment variables of the module chart.
The first category uses the format <variable_name>=<value> and is represented by the following files:
| Configuration file | Variable name | Description |
| ./cfg/smc.settings.cfg | apiserver_advertise_address | address for kube-apiserver, usually the machine's internal IP address. |
| external_ip_address | address for ingress-controller, must specify the machine's external IP address. | |
| ./cfg/license-server.settings.cfg | license_key | license key. Typically, the key is sent with the accompanying distribution letter. If the key is missing, contact your sales manager. |
| license_server_address | license server address. |
The second category uses YAML format. These settings are subsequently applied directly during chart installation as external values to override the internal values of the Helm chart. Each chart has a complete standard set of environment variables. Only those variables that need to be changed are included in the external values files.
For changing most environment variables, a flexible mechanism is provided that allows passing YAML objects directly to the final deployment template.
For services that support this configuration, the envOverride field is used. In this field, in the form of a YAML dictionary, the key is the variable name, and the value is a YAML object that will be inserted into the final template as the service's environment variable.
Example:
envOverride:
NO_EMAIL_CHECK_ON_REGISTRATION:
value: 'true'
Variable values must always be strings. Otherwise, an error will occur during deployment.
The flexible environment variable customization capability is available in the following files:
| Configuration file | Configuration section | Variable name | Default value | Description |
| ./cfg/video-recorder.values.yaml | video_recorder | UVICORN_CONCURRENCY | value: '30' | number of simultaneously processed HTTP connections per pod. Exceeding the limit will return a 503 error. |
| SQLALCHEMY_POOL_SIZE | value: '20' | maximum number of persistent connections that the SQLAlchemy pool will maintain per pod. | ||
| SQLALCHEMY_POOL_OVERFLOW | value: '10' | number of connections above the persistent limit that will be closed after use per pod. | ||
| RETENTION_BATCH_SIZE | value: '83' | size of the batch of attempt video content to be deleted at one time. | ||
| RETENTION_BATCH_PAUSE | value: '4' | time in seconds between batch deletions. | ||
| RETENTION_CHECK_FREQUENCY | value: '3600' | frequency of cleanup check execution in seconds. | ||
| minio | S3_ERROR_DATA_RETENTION_DAYS | value: '3' | retention period for error data in days in the object storage. | |
| MINIO_API_DELETE_CLEANUP_INTERVAL | value: '30s' | interval at which MinIO deletes objects from the bucket. | ||
| S3_BUCKETS_PREFIX | value: 'vr' | prefix placed before all bucket names in the following format: <prefix>.<bucket name> | ||
| ./cfg/platform.values.yaml | backend | QUERY_LIMIT | value: '100' | limit on the number of returned elements in API requests for retrieving system entities. Increasing this limit is not recommended as the API request execution time may increase several times. Also note that increasing limits will degrade system performance. |
| INDEX_UPDATE_PERIOD | value: '60' | value in seconds describing how long after addition a profile will appear in the search index. The default value is 60 seconds. To increase the search index update speed, reduce this value. This value affects authorization speed after registration. | ||
| PLANNED_RETENTION_POLICY_DAY_OF_WEEK | value: '*' | day of the week in cron format when the scheduled cleanup task will be executed. | ||
| PLANNED_RETENTION_POLICY_HOUR | value: '*' | hour in cron format when the scheduled cleanup task will be executed. | ||
| PLANNED_RETENTION_POLICY_MINUTE | value: '59' | minute in cron format when the scheduled cleanup task will be executed. | ||
| DELETE_SAMPLE_BATCH_SIZE | value: '1000' | size of the attempt frame batch to be deleted at one time. | ||
| DELETE_SAMPLE_BATCH_SLEEP | value: '0' | time in seconds between batch deletions. | ||
| SAMPLE_TTL | value: '315360000' | time in seconds after which attempt frames will be deleted. | ||
| matcher | MATCHER_SEMAPHORE_SIZE | value: '1' | interface parameter that regulates the number of concurrent calls to matcher-service. Must equal the shard.quantity parameter in the matcher.values.yaml file. | |
| ./cfg/matcher.values.yaml | shard | SEARCH_THREADS_N | value: '1' | number of threads used to perform the search. |
| ./cfg/baf.values.yaml | baf | POOL_SIZE | value: '100' | connection pool size. It may be necessary to increase this value if high load on the BAF services is expected. |
| ./cfg/report.values.yaml | report | REPORT_PROCESS_COUNT | value: '5' | number of report generation processes. The more processes, the more reports can be generated simultaneously. |
There are also classic variables that are substituted into the template by value. This is either due to the complex logic of their use in charts or the lack of need for deep customization.
Values written with dots in YAML files indicate a nested structure.
List of files with classic variables:
| Configuration file | Variable path | Description |
| ./cfg/platform.values.yaml | ingress.enable | set this parameter to 1 to provide external access to the OMNI Platform module via an Ingress resource. The Ingress will use the domain from the ingress.rules.gateway.host parameter. |
| ingress.rules.gateway.host | domain name used in the Ingress to route requests to Kubernetes services for OMNI Platform. | |
| postgres.enable | disable if using your own database server. Note: You must change the values for postgres-root-credentials in the ./cfg/platform.secrets.json file and the values for postgres.host and postgres.port in the current file. | |
| minio.enable | indicates that object storage will be used for storing images from events. If disabled, PostgreSQL will be used for storing images. Note: If a certain number of images have already accumulated after the first deployment, it is not recommended to disable this parameter, otherwise it will be impossible to access the accumulated data. When deploying in AWS, this parameter must be disabled. | |
| backend.retention_workers_count | number of processes performing attempt frame cleanup. | |
| ./cfg/matcher.values.yaml | shard.quantity | number of shards created. Increasing this parameter speeds up search on large datasets. |
| ./cfg/baf.values.yaml | ingress.rules.gateway.host | domain name used in the Ingress to route requests to Kubernetes services for BAF. |
| gateway.dns_resolver | domain name of the DNS server that will be used for resolving Kubernetes service names. | |
| gateway.dns_validity_time | time after which the domain name resolution will be performed again. | |
| ./cfg/platform-ui.values.yaml | ingress.host | domain name used in the Ingress to route requests to Kubernetes services. The value must match the ingress.rules.gateway.host value in the ./cfg/platform.values.yaml file. |
| ./cfg/video-recorder.values.yaml | minio.enable | disable if using your own MinIO server. Note: You must change the values for video-recorder-minio in the ./cfg/video_recorder.secrets.json file and the values for minio.host, minio.port and minio.secure (1 for https, 0 for http) in the current file. |
| video_recorder.decoding_enabled | if you don't need video decoding, set to 0. | |
| video_recorder.decoding_process_count | number of decoding processes launched in one service replica. The more processes, the more simultaneous video decoding sessions the service will handle. | |
| video_recorder.retention_workers_count | number of processes performing attempt video content cleanup. | |
| ./cfg/util-services.values.yaml | ingress.rules.gateway.host | must equal the ingress domain in the ./cfg/baf.values.yaml file. |
| ./cfg/image-api.values.json | processing.enable_ingress | set this parameter to 1 to provide external access to the image-api module via an Ingress resource. The Ingress will use the domain from the ingress.host parameter. |
| ingress.host | domain name used in the Ingress to route requests to Kubernetes services for the image-api module. |
The third category is stored in JSON files and is used by delivery scripts to create secrets inside the Kubernetes cluster.
If you need to use your own secrets, you must add to envOverride (in the values.yaml file of the corresponding module) a variable with the name of the variable associated with the secret, and describe in it the correct YAML for obtaining data from your secret.
List of secret files and their environment variables:
| Configuration file | Secret name | Secret description | Secret field | (Values.yaml filename) Section\Environment variable name |
| ./cfg/platform.secrets.json | license-secret | license key and license server address. | server_url | licensing\LIC_SERVER_URL |
| key | licensing\LIC_KEY | |||
| rabbit-secret | username and password for accessing the message broker, used for internal interaction of OMNI Platform services. Set an arbitrary name consisting of Latin letters without spaces, and a password consisting of Latin letters and digits without spaces. | rabbit_user | rabbitmq\RABBITMQ_DEFAULT_USER | |
| rabbit_password | rabbitmq\RABBITMQ_DEFAULT_PASS | |||
| postgres-root-credentials | username and password for the root user in the database. | user |
| |
| password |
| |||
| platform-service-key | secret key for internal interaction of OMNI Platform services | service_key |
| |
| platform-user-secret | credentials (email address and password) that will be used to access OMNI Platform. Upon first deployment, users will be created automatically. The email address must consist of digits, Latin letters and symbols. The admin user has system administrator rights. The default user can be used for unprivileged access to the system. | admin_email | backend\PLATFORM_ADMIN_EMAIL | |
| admin_password | backend\PLATFORM_ADMIN_PASSWORD | |||
| default_email | backend\PLATFORM_DEFAULT_EMAIL | |||
| default_password | backend\PLATFORM_DEFAULT_PASSWORD | |||
| platform-email-secret | SMTP server parameters. To disable email sending, leave the fields empty. | email_host | backend\EMAIL_HOST | |
| email_port | backend\EMAIL_PORT | |||
| email_from | backend\EMAIL_FROM | |||
| email_host_user | backend\EMAIL_HOST_USER | |||
| email_host_password | backend\EMAIL_HOST_PASSWORD | |||
| email_use_ssl | backend\EMAIL_USE_SSL | |||
| minio-credentials | username and password for the root user in the object storage. | root-user | event_service\S3_ACCESS_KEY | |
| root-password | event_service\S3_SECRET_KEY | |||
| storage-engine-postgres | username, password and database name for platform. | user | backend\POSTGRES_USER | |
| password | backend\POSTGRES_PASSWORD | |||
| db | backend\POSTGRES_DB | |||
| event-service-postgres | username, password and database name for event-service. | user | event_service\POSTGRES_USER | |
| password | event_service\POSTGRES_PASSWORD | |||
| db | event_service\POSTGRES_DB | |||
| ./cfg/baf.secrets.json | baf-postgres | database connection data. | user |
|
| password |
| |||
| db |
| |||
| baf-email-secret | SMTP server connection data. | host | backend\Email__Host | |
| port | backend\Email__Port | |||
| from | backend\Email__From | |||
| host_user | backend\Email__User | |||
| host_password | backend\Email__Password | |||
| use_ssl | backend\Email__UseSsl | |||
| ./cfg/video-recorder.secrets.json | video-recorder-tokens | access token and encryption token. WARNING! The video-recorder-tokens fields are filled during installation. | access_token | video_recorder\TOKEN |
| fernet_key | video_recorder\FERNET_KEY | |||
| video-recorder-postgres | database connection data. | user | video_recorder\POSTGRES_USER | |
| password | video_recorder\POSTGRES_PASSWORD | |||
| db | video_recorder\POSTGRES_DB | |||
| video-recorder-minio | database connection data. | user | minio\MINIO_ROOT_USER | |
| password | minio\MINIO_ROOT_PASSWORD |
Updating configuration of a running instance
To apply updates to module values, run the module installation command. If the update doesn't take effect, uninstall and reinstall the module.
For updates involving module resources (e.g., secrets), execute the module resource installation command. Then, restart the module pods using Kubernetes tools to ensure they pick up the changes.
Extended configuration (optionally)
Docker settings for GPU usage
To set nvidia-container-runtime as the default low-level runtime, run the following command:
./cli.sh smc nvidia install
Configuring GPU usage in BAF
To enable GPU in BAF, you need to override the CONFIGS variable for the face-detector-template-extractor service. This is done in the ./cfg/image-api.values.yaml file.
In the processing.services section, add the following YAML object:
face-detector-template-extractor:
envOverride:
CONFIGS:
value: '{"capturer": {"name": "common_capturer_uld_fda.xml", "params": {"use_avx2": 0, "downscale_rawsamples_to_preferred_size": 0}}, "recognizer": {"name": "method12v1000_recognizer.xml", "params": {"use_avx2": 0, "use_cuda": 1}}}'
Install and configure a cluster
If you already have a deployed cluster, move to Configure licensing section.
To create and configure the cluster, run the following commands:
./cli.sh smc system-patch
./cli.sh smc install
./cli.sh platform db-create-mountpoint
./cli.sh platform install-secrets
These commands:
- Create database mount point.
- Initialize a cluster.
- Install the secrets.
To use GPU in a cluster, install NVIDIA device plugin:
$ ./cli.sh smc nvidia install
Cluster health check
After initializing the master node, make sure that all nodes are ready for operation and have the Ready status. You can check this by running the command below:
$ kubectl get nodes
As a result, the following output will be displayed in the terminal:
NAME STATUS ROLES AGE VERSION
master-node Ready control-plane,master 11d v1.23.8
To check all cluster components, run the following command:
$ kubectl get all --all-namespaces
Configure licensing
The user has 3 license activation options: trial period activation, online and offline license activation.
Install and run a license server
Before installation, open the license-server.settings.cfg file and set the IP address of the machine, on which the license server will be installed, in the license_server_address field.
Run the command below to install the license server. If license_server_address differs from the host address of the machine where the deployment is taking place, it will be set via sshpass.
$ ./cli.sh license-server install
Check that the license server is in the Running status:
$ ./cli.sh license-server status
Console output example:
floatingserver.service - Floating License Server
Loaded: loaded (/etc/systemd/system/floatingserver.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2022-12-20 12:25:54 +05; 1min 48s ago
To check that the license server is available, follow http://<license_server_address>:8090 in your web browser. As a result, you should be redirected to the login form.
Trial period activation
Please note that:
- The Internet connection is required.
- Running on a virtual machine is not allowed.
The trial period is activated the first time you launch BAF.
Online license activation
Before activation, make sure that the key field (from file ./cfg/license-server.settings.cfg) contains the license key.
Run the license activation command:
$ ./cli.sh license-server activate
When license is successfully activated, the console will return the following result:
[2022-12-20 12:25:53+05:00] INF Activating license key...
[2022-12-20 12:25:54+05:00] INF License activated successfully!
Offline license activation
Before activation, make sure that the license_key field (from file ./cfg/license-server.settings.cfg) contains the license key.
For offline activation, set "1" in the enable_offline_activation field in the license-server.settings.cfg file.
Run the command below to generate an offline license request:
$ ./cli.sh license-server generate-offline
As a result, the request-offline.dat file should appear in the setup directory.
Send the generated request-offline.dat request file to baf-request@3divi.com. The license file will be submitted in the response email.
Copy the received license file to the setup folder.
Open the configuration file license-server.settings.cfg in a text editor and fill in the variable license_offline_activation_file with the license file name and its extension, if present, separated by a dot.
Run the license activation command:
$ ./cli.sh license-server activate
When license is successfully activated, the console will return the following result:
[2022-09-08 01:30:36+05:00] INF Offline activating license key...
[2022-09-08 01:30:36+05:00] INF License activated successfully!
Deployment
Launch deployment
Install facial recognition subsystem (OMNI Platform)
Run the installation of the first OMNI Platform module:
$ ./cli.sh image-api install
Install matcher (matcher-router + matcher-shard):
./cli.sh matcher install
Run the installation of the second OMNI Platform module:
$ ./cli.sh platform install
If necessary, run the installation of OMNI Platform web interface:
./cli.sh platform-ui install
For the platform-ui module to work, you need to enable the Ingress for the OMNI Platform module (parameter ingress.enable in the ./platform.values.yaml file).
To continue the installation, open the /etc/hosts file and add the following lines at the end of the file:
<external_ip_address> : <platform_domain>
<external_ip_address> : <baf_domain>
Install the Video Recorder (VR) subsystem (optional)
If you are updating from version 1.12.0, make sure to transfer the motion control patterns, if necessary, to the settings of the new version. If you do not do this before installing the new version of the video recorder, the pattern data will be irretrievably lost.
For detailed information, see the Update Instructions section.
If you do not need to record video, skip this step
In the Registration by Selfie scenario Video Recorder provides the ability to save video attempts and perform Motion Control.
Create a directory to store the object store data using the command:
$ ./cli.sh video-recorder minio-create-mountpoint
Run the command to generate Video Recorder tokens:
$ ./cli.sh video-recorder generate-token
Example of output to the console:
sha256:2473ba0ebf5ef66cd68b252bba7b46ae9f7cc3657b5acd3979beb7fbc5d8807f
Fernet key: ......
Access token: ......
As a result, you will get two tokens that need to be written in the configuration file ./cfg/video-recorder.secrets.json in the section video-recorder-tokens.
Run the command to install the Video Recorder secrets:
$ ./cli.sh video-recorder install-secrets
Run the command to install the Video Recorder:
$ ./cli.sh video-recorder install
Install BAF
The pattern mechanism has been updated in version 1.13.0. Please refer to updates to transfer patterns between versions.
Run the command to install BAF secrets for the cluster:
$ ./cli.sh baf install-secrets
Run BAF installation:
$ ./cli.sh baf install
To monitor the deployment process, open another terminal tab and enter the following command:
$ watch 'kubectl get pods'
To use the system, you need to create a user. For more information, see User management instructions section.
Install the Report Service (RS) reporting subsystem (optional)
The RS subsystem is needed for generating reports on system performance. If reporting functionality is not required, you can skip this step.
To install the Report Service, run the following command:
$ ./cli.sh report install
Install the Util Services (US) subsystem (optional)
The US subsystem is required to run optional services for monitoring BAF operations.
To install Util Services, run the command below:
$ ./cli.sh util-services install
Configure DNS server
To provide access to BAF, DNS server on your network should contain a record that domain is available at <external_ip_address>.
For testing you need to fill in IP address and domain in the /etc/hosts file on Linux or C:\Windows\System32\drivers\etc\hosts on Windows.
To do this, add a new line like <external_ip_address> <host> at the end of this file, set the values for the corresponding variables and save the file. Note that you need to have administrator privileges to edit the hosts file.