Installation instructions

Upload images and create a Kubernetes cluster

Preparation

Download and extract OMNI Platform distribution kit to the machine used for installation. To do this, use the command below:

curl --output platform.zip link to the distribution kit

In the curl request, specify a link to the platform distribution kit (zip file). A link to the folder that contains the distribution kit and accompanying documentation in pdf format will be provided in the email.

Next, move the face_sdk.lic license file (attached to the email) to the setup folder.

The distribution kit contains:

• ./cli.sh: entry point to run the commands
• ./cfg: folder with OMNI Platform configuration files

Further commands are to be executed in the system console from setup directory.

Upload images

First, upload OMNI Platform images from the archive to the local registry:

./cli.sh generic load-images

Then, upload the infrastructure images from the archive to the local registry:

./cli.sh smc load-images

Uploading can last for about five minutes.

Configuration

Enter environment variables

Open the configuration files below in a text editor, set values for the variables and save changes.

note

Values set in yaml files via dots denote nesting.

Configuration File	Variables
./cfg/smc.settings.cfg	apiserver_advertise_address: address of the kube-apiserver. In most cases this is the machine's internal IP address. external_ip_address: address for the ingress-controller. Here you need to specify the machine's external IP address.
./cfg/license-server.settings.cfg	license_key: this key is usually sent in the accompanying email to the distribution kit. To get the key, contact your sales manager. This key is not required for the trial period. license_server_address: IP address of the machine, on which the license server will be installed
./cfg/platform.secrets.json	license-secret : here you need to set a license key and address of the license server instead of license_server_address. docker-registry : values to specify if external docker-registry is used rabbit-secret: user name and password to get access to a message broker, used for internal interaction of OMNI Platform services. Set an arbitrary name that consists of Latin letters without spaces and a password that consists of Latin letters and numbers without spaces. postgres-root-credentials: user name and password for root user in the database platform-service-key: secret key required for internal communication between OMNI Platform services platform-user-secret: user credentials (email address and password) used for access to OMNI Platform. Users will be created automatically on the first deployment. The specified email address must include numbers, Latin letters and symbols. platform-email-secret: SMTP server settings. To disable sending emails, leave these fields blank. minio-credentials: username and password for root user in object storage
./cfg/platform.values.yaml	backend.query_limit: this value limits the number of returned elements in API requests to get system objects. Increasing this limit is not recommended, as API request runtime may increase several times. Also, please note that increasing the limits will lead to the system degradation. backend.index_update_period: this value in seconds describing the amount of time the added profile will appear in the search index. The default value is 60 seconds. To speed up the index updating, decrease the parameter value. backend.enable_profile_autogeneration: auto-generation of profiles for incoming activities from the agent. Please note that enabling this option will increase the consumption of license resources (database size). To enable this function, set the value to 1. processing.use_cuda: responsible for use graphics card in processing service. 0 – disabled, 1 – enabled. ingress.rules.gateway.host: domain name or IP address used in ingress to route requests to Kubernetes services postgres.enable: disable if using your own database server. You need to change the values for postgres-root-credentials in the ./cfg/platform.secrets.json file and the values for postgres.host and postgres.port in the current file. minio.enable: enabling means that object storage will be used to store images. If you disable this option, postgres will be used to store images. Note: If after the first deployment a certain number of images have already been collected, you should not disable this option, since then you won't be able to obtain this data. This option must be disabled for AWS deployment. minio.buckets_prefix: specifies the prefix for all buckets that will be used by the module in MinIO in the format <prefix>.<bucket name>. If a prefix is not required, leave the field empty. Note: If you need to preserve data in MinIO from the previous version, also leave the field empty. matcher.shard_n: interface parameter, regulates the number of simultaneous calls to matcher-serivice. Must be equal to the shard.quantity parameter in the matcher.values.yaml file event_service.pool_size: the maximum number of active and inactive connections in the pool that a single worker can use. event_service.max_overflow: the maximum number of temporary connections that can be opened beyond the value specified in the event_service.pool_size parameter. For more details, see a Database connections section. event_service.pool_recycle: the time in seconds after which connections in the pool should be reopened. event_service.pool_pre_ping: a parameter that enables or disables the database connection check before sending requests. Note: This parameter controls the service-side connection check; the database-side handling is configured separately. event_service.ws_timeout: the time in seconds after which a WebSocket connection will be closed during interruptions. event_service.bucket_names: the names of the buckets that the event service will use. Note: Do not change the bucket names if you need to preserve data from the previous version.
./cfg/matcher.values.yaml	shard.quantity: the number of shards to be created; increasing the parameter allows you to speed up searches on large data shard.search_threads_n: the number of threads used to perform the search
./cfg/platform-ui.values.yaml	ingress.host: domain name, used in ingress to route requests to Kubernetes services. The value must match the value of ingress.rules.gateway.host in the ./cfg/platform.values.yaml file.

Docker settings for GPU usage (optionally)

• Docker configuration

  To set nvidia-container-runtime as the default low-level runtime, add the following lines to the configuration file located at /etc/docker/daemon.json:

  "default-runtime": "nvidia",
  "runtimes": {
    "nvidia": {
      "path": "/usr/bin/nvidia-container-runtime",
      "runtimeArgs": []
      }
  }

• Applying configuration

  To apply configuration, restart docker-service:

  sudo systemctl restart docker

Install and configure a cluster

note

If you already have a deployed cluster, move to Launch Deployment.

To create and configure the cluster, run the following commands:

./cli.sh smc system-patch
./cli.sh smc install
./cli.sh platform db-create-mountpoint
./cli.sh platform install-secrets

This commands:

• Create database mount point.
• Initialize a cluster.
• Install the secrets.

note

To use GPU in a cluster, install NVIDIA device plugin:

./cli.sh smc nvidia install

Cluster health check

After initializing the master node, make sure that all nodes are ready for operation and have the Ready status. You can check this by running the command below:

kubectl get nodes

As a result, the following output will be displayed in the terminal:

NAME          STATUS      ROLES                   AGE     VERSION
master-node   Ready       control-plane,master    11d     v1.23.8

To check all cluster components, run the following command:

kubectl get all --all-namespaces

Configure licensing

Trial activation

The trial period is activated the first time you launch OMNI Platform.

Attention

• The Internet connection is required.
• Installing a license server on a virtual machine is not allowed.

Install a license server

Before installation, open the ./cfg/license-server.settings.cfg file and set the IP address of the machine, on which the license server will be installed, in the license_server_address field.

Run the command below to install the license server. If license_server_address differs from the host address of the machine where the deployment is taking place, it will be set via sshpass.

./cli.sh license-server install

Run the command to start the license server:

./cli.sh license-server start

Check that the license server is in the running status:

./cli.sh license-server status

Console output example:

floatingserver.service - Floating License Server
    Loaded: loaded (/etc/systemd/system/floatingserver.service; enabled; vendor preset: enabled)
    Active: active (running) since Tue 2022-12-20 12:25:54 +05; 1min 48s ago

To check that the license server is available, follow http://<license_server_address>:8090 in your web browser. As a result, you should be redirected to the login form.

Online license activation

Before activation, make sure that the key field (from file ./cfg/license-server.settings.cfg) contains the license key.

Run the license activation command:

./cli.sh license-server activate

When license is successfully activated, the console will return the following result:

[2022-12-20 12:25:53+05:00] INF Activating license key...
[2022-12-20 12:25:54+05:00] INF License activated successfully!

Offline license activation

Before activation, make sure that the license_key field (from file ./cfg/license-server.settings.cfg) contains the license key.

note

For offline activation, set "1" in the enable_offline_activation field in the license-server.settings.cfg file.

Run the command below to generate an offline license request:

./cli.sh license-server generate-offline

As a result, the request-offline.dat file should appear in the setup directory.

Send the generated request-offline.dat request file to support-platform@3divi.com. The license file will be submitted in the response email.

Copy the received license file to the setup folder.

Open the configuration file license-server.settings.cfg in a text editor and fill in the variable license_offline_activation_file with the license file name and its extension, if present, separated by a dot.

Run the license activation command:

./cli.sh license-server activate

When license is successfully activated, the console will return the following result:

[2022-09-08 01:30:36+05:00] INF Offline activating license key...
[2022-09-08 01:30:36+05:00] INF License activated successfully!

Check the license status after activation

Run the command to get the license status:

./cli.sh license-server status-license

Example console output:

Activation Status: OK
License key: YOUR-LICENSE-KEY
Days Left To Expiration: 100

note

Compare the command output with your license information. Days Left To Expiration=14 means that the trial license was automatically activated. In this case, repeat the activation steps.

Deployment

Launch deployment

Install Image API:

./cli.sh image-api install

Install Matcher (matcher-router + matcher-shard):

./cli.sh matcher install

To install OMNI Platform, run the following script:

./cli.sh platform install

If necessary, run the installation of OMNI Platform web interface:

./cli.sh platform-ui install

To monitor the deployment progress, open another terminal tab and enter the following command:

watch 'kubectl get pods'

OMNI Platform is running if all pods have the Running status.

note

After startup, check the license status. If the received data is not true, insert the license key into the fields license_key (in ./cfg/license-server.settings.cfg) and license-secret.key (in ./cfg/platform. secrets.json) and perform all the steps from here.

Configure DNS server

To provide access to OMNI Platform, DNS server on your network should contain a record that domain is available at <external_ip_address>.

For testing you need to fill in IP address and domain in the /etc/hosts file on Linux or C:\Windows\System32\drivers\etc\hosts on Windows. Just add a new line like <external_ip_address> <ingress.rules.gateway.host> to the end of the file and save the file. The variable values are specified in the ./cfg/smc.settings.cfg and ./cfg/platform.values.yaml files, respectively.

Note that you need to have administrator privileges to edit the hosts file.

To use OMNI Platform at the machine where it was deployed, you can use a script below. It will automatically add the necessary entry to the /etc/hosts file.

./cli.sh generic add-dns - <external_ip_address> <domain>

Description of deployed system

To get the status of OMNI Platform services, run the command below:

kubectl get pods

As result, the console will display a list of services, their statuses, the number of restarts, and the service age.

The example of console output:

NAME                                                        READY          STATUS         RESTARTS         AGE
image-api-age-estimator-dep-c66c8f575-zb84s                  1/1           Running         0               24h
image-api-body-detector-dep-5588d96ddc-lnrrx                 1/1           Running         0               24h
image-api-emotion-estimator-dep-b6c947ff-mz7cw               1/1           Running         0               24h
image-api-face-detector-face-fitter-dep-                     1/1           Running         0               24h
image-api-face-detector-liveness-estimator-dep-              2/2           Running         0               24h
image-api-face-detector-template-extractor-dep-              1/1           Running         0               24h
image-api-gender-estimator-dep-7948d8cf85-4hk2q              1/1           Running         0               24h
image-api-mask-estimator-dep-5ccfcc8cb9-sjnqz                1/1           Running         0               24h
image-api-quality-assessment-estimator-dep-                  1/1           Running         0               24h
image-api-verify-matcher-dep-6b8f5b4d6f-jjqr4                1/1           Running         0               24h
image-api-template-extractor-dep-64d9d7f6f7-b5bpd            1/1           Running         0               24h
image-api-liveness-estimator-dep-6b757b994c-hd9cn            2/2           Running         0               24h
platform-activity-matcher-dep-8697574c9b-xvh5z               2/2           Running         0               24h
platform-agent-sync-dep-899c9ddc8-7qcvj                      1/1           Running         0               24h
platform-backend-dep-685675d648-ngcrj                        1/1           Running         0               24h
platform-event-service-dep-5fcd66f999-24jjg                  1/1           Running         0               24h
platform-admin-static-dep-f4cd7db78-h86jz                    1/1           Running         0               24h
platform-licensing-dep-84d78745d6-2flk2                      1/1           Running         0               24h
platform-memcached-dep-64d9d7f6f7-b5bpd                      1/1           Running         0               24h
platform-minio-dep-58fc7db749-l457d                          1/1           Running         0               24h
platform-postgres-dep-67c5b75b84-fm8wr                       1/1           Running         0               23h
platform-rabbit-dep-69cd659f8c-sbnk2                         1/1           Running         0               24h
platform-redis-dep-694df659f-9vhrh                           1/1           Running         0               24h
platform-ui-dep-78bcb9d878-8mtqx                             1/1           Running         0               24h
platform-matcher-adapter-dep-dd947d74c-5cjz7                 1/1           Running         0               24h
matcher-router-dep-76b9766867-fcrtl                          1/1           Running         0               24h
matcher-shard-dep-0-5c5c786694-wqvpk                         1/1           Running         0               24h

Overview of the services is given below:

• platform-activity-matcher-dep is the service used to search for people by activities.
• image-api-age-estimator-dep is the service used to estimate a person’s age from a face image.
• platform-agent-sync-dep is the service responsible for synchronization of profile data with OMNI Agents.
• platform-backend-dep is the main container of OMNI Platform, responsible for how most of API works.
• image-api-body-detector-dep is the service designed to detect bodies in an image.
• platform-rabbit-dep is RabbitMQ service used to work with asynchronous task queue.
• platform-cache-dep is Memcached service used for data caching.
• platform-postgres-dep is an instance of PostgreSQL database that stores all information of OMNI Platform.
• platform-minio-dep is MinIO service used to store images.
• image-api-emotion-estimator-dep is the service that estimates a person's emotions from a face image.
• image-api-face-detector-face-fitter-dep is the service used to determine the anthropometric points of the face and the head rotation angles.
• image-api-face-detector-liveness-estimator-dep is the service used to detect a face and determine if the face in the image is real or fake.
• image-api-face-detector-template-extractor-dep is the service used to detect faces and extract biometric templates.
• image-api-liveness-estimator-dep is the service used to determine if the face in the image is real or fake.
• platform-admin-static-dep is the service responsible for the web interface of Django administration page.
• image-api-gender-estimator-dep is the service used to estimate a person’s gender from a face image.
• platform-licensing-dep is the service that limits the Platform capabilities according to the license parameters.
• image-api-mask-estimator-dep is the service that detects if a person in the image is wearing a medical mask.
• image-api-quality-assessment-estimator-dep is the service designed to assess the quality of the face image.
• platform-redis-dep is Redis service used to work with webSockets.
• image-api-verify-matcher-dep is the service that compares two face images to determine if they belong to the same person.
• platform-event-service-dep is the service used to handle events coming from OMNI Agent.
• platform-ui-dep is the service responsible for OMNI Platform web interface.
• matcher-adapter-dep is the service responsible for synchronizing the platform’s face database and the Matcher Service search index.
• matcher-router is the service used for balancing face search requests on matcher-shards.
• matcher-shard is the service responsible for searching for faces.