Processes and events

OMNI Agent converts the recognition and tracking data into the formats of processes and events for subsequent sending to OMNI Platform or third-party services.

Processes

A data format that represents recognition and tracking results within time intervals.

Each process contains the following set of data:

• Process ID
• Process time interval (start and end of the process)
• Process type (track, human, face, body, emotion etc.)
• Detection object with face attributes (object is a human in the camera’s field of view)
• Best shot

Processes are sent to OMNI Platform via the HTTP protocol or to a third-party service via webhook / MQTT.

Send processes via webhooks

You can add a webhook in Settings of OMNI Agent dashboard or open the config/run_params.json configuration file and specify values for the webhook_tracking_subscribers parameter. This parameter represents a list of objects to determine the address of the server that receives data from OMNI Agent and is available in two versions:

Version 1

{
    "url": "http://127.0.0.1:5000/trigger" // address with the endpoint
}

Version 2 (default)

{ 
    "host": "127.0.0.1", // address
    "port": "5000", // port
    "is_secured": false, // http/https flag
    "path": "/trigger" // endpoint
}

An array of processes is sent every 5 seconds (by default) as well as at the beginning and end of tracking, when a person is matched with a database, when a person gets into ROI or crosses marked lines. To change the process transfer interval, edit the ongoings_interval_in_msec field in the config/run_params.json file (measured in milliseconds).

To send bestshots via webhooks, enable the "Send bestshots in webhook" option in the Settings of the OMNI Agent web interface, or specify the value true for the enable_webhook_image2jpg_conversion field. The default value is false. The bestshot is encoded in base64 as a jpeg image.

Send processes via MQTT

1. Download and install the MQTT brocker from here

2. Open the config/run_params.json file and enter the values for the following fields:

   • mqtt_settings – parameters for connecting to the MQTT broker
     • enable – bool
     • brocker_address – string, address of the MQTT broker that receives processes. The current implementation will only work with localhost
     • mqtt_port – int, the port on which the MQTT broker listens for connections
     • client_identifier – string, unique MQTT client identifier that will be used when connecting to the broker
     • mqtt_topic – string, the topic in the MQTT broker to which processes will be sent

   Example of filled fields:

   "mqtt_settings": {
   "brocker_address": "localhost",
   "client_id": "default_mqtt_client",
   "mqtt_port": 1883,
   "mqtt_topic": "events",
   "enable": true
   }

3. Start the Mosquitto MQTT service (on Linux system):

   sudo /etc/init.d/mosquitto start

   Basic commands:

   • mosquitto_sub: a command in the MQTT message broker that is used to subscribe to topics ("mqtt_topic") and receive messages published to those topics.
   • -h <hostname>: specifies the hostname (or IP address) of the MQTT broker to connect to.
   • -p <port>: specifies the port on which the MQTT broker listens (default 1883).
   • -t <topic>: specifies the topic you want to subscribe to.

   Usage example:

   mosquitto_sub -h localhost -p 1883 -t events

   This command will connect to the MQTT broker on localhost, subscribe to the "events" topic, and print received messages to the screen.

Events

A data format that represents recognition and tracking results as facts. OMNI Agent can generate the following types of events:

• Identification/non-identification events (a person in the camera's field of view is identified/non-identified)
• ROI events (a person is in/out of ROI)
• Line crossing events (a person crossed the line marked on the video stream in the forward/reverse direction)
• HAR events (a person has fallen, sits, lies or is involved in a fight)

Event structures

Structure of Identification Event:

• type – identification
• date – event time
• id – event identifier
• object (the object that generated the event):
  • class – object class (for example, human)
  • id – object identifier
• parents (event parent, usually a process):
  • id – event parent identifier
  • type – type of the event parent (for example, human, face)
• source – video stream identifier
• trigger_source:
  • id
• image – cropped image
• original_image – full frame
• identification_data – array of identification candidates (profiles):
  • profile_id – profile identifier
  • group_ids – identifier of the group to which the profile was added
  • score – degree of similarity of the face from the frame with the face from the profile from 0 (0%) to 1 (100%)
  • far – False Acceptance Rate when the system mistakes images of different people as images of the same person
  • frr – False Rejection Rate when the system mistakes two images of the same person as images of different people
  • distance – distance between compared template vectors. The smaller the value, the higher the confidence in correct recognition

Structure of Non-Identification Event:

• type – non_identification
• date – event time
• id – event identifier
• object (the object that generated the event):
  • class – object class (for example, human)
  • id – object identifier
• parents (event parent, usually a process):
  • id – event parent identifier
  • type – type of the event parent (for example, human, face)
• source – video stream identifier
• trigger_source:
  • id
• image – cropped image
• original_image – full frame

Structure of ROI Event:

• type – roi
• date – event time
• id – event identifier
• parents (event parent, usually a process):
  • id – event parent identifier
  • type – type of the event parent (for example, human, body)
• source – video stream identifier
• trigger_source:
  • id
• image – cropped image
• original_image – full frame
• roi_data:
  • direction (a person is in/out of ROI) – in/out

Structure of Line Crossing Event:

• type – crossing
• date – event time
• id – event identifier
• parents (event parent, usually a process):
  • id – event parent identifier
  • type – type of the event parent (for example, human, body)
• source – video stream identifier
• trigger_source:
  • id
• image – cropped image
• original_image – full frame
• crossing_data:
  • direction (a person crossed the line the forward/reverse direction) – in/out

Structure of HAR Event:

• type – har
• date – event time
• id – event identifier
• parents (event parent, usually a process):
  • id – event parent identifier
  • type – type of the event parent (for example, human, body)
• source – video stream identifier
• trigger_source:
  • id
• image – cropped image
• original_image – full frame
• har_data:
  • action – fight, fall, sit, lie

Events are sent to OMNI Platform via the HTTP protocol. OMNI Platform can transfer the received events to a third-party service via web sockets. Fot more information see OMNI Platform documentation.