Face, Body and Object detection

In this section you will learn how to integrate Face Detector, Human Body Detector and Object Detector to your C++ or Python project.

Face Detection (С++/Python)

Requirements

• Windows x86 64-bit or Linux x86 64-bit system.
• Installed Face SDK package windows_x86_64 or linux_x86_64 (see Getting Started).

1. Creating a Face Detector

1.1 To create a Face Detector, follow steps 1-3 described in Creating a Processing Block and specify the values:

• "FACE_DETECTOR" for the "unit_type" key;
• path to Face Detector model file for the "model_path" key.

C++

configCtx["unit_type"] = "FACE_DETECTOR";

// default path to Human Body Detector model file - "share/facedetectors/face.enc" in the Face SDK's root directory
configCtx["model_path"] = "share/facedetectors/face.enc";

// optional, default value is 0.5
configCtx["confidence_threshold"] = 0.5;

// optional, default value is 0.5
configCtx["iou_threshold"] = 0.5;

Python

configCtx = {
    "unit_type": "FACE_DETECTOR",
    # the path is relative to the Face SDK root directory
    "model_path": "share/facedetectors/face.enc",

    # optional, default values are specified after ":"
    "confidence_threshold": 0.5,
    "iou_threshold": 0.5
}

1.2 Create a Face Detector Processing block:

C++

pbio::ProcessingBlock faceDetector = service->createProcessingBlock(configCtx);

Python

faceDetector = service.create_processing_block(configCtx)

2. Face Detection

2.1 Create a Context container ioData for input-output data using the createContext() method:

auto ioData = service->createContext();

2.2 Create a Context container imgCtx with RGB-image following the steps described on Creating a Context container with RGB-image.

# copy an image into the binary format
input_rawimg = image.tobytes()
# put an image into the container
imageCtx = {
    "blob": input_rawimg,
    "dtype": "uint8_t",
    "format": "NDARRAY",
    "shape": [dim for dim in img.shape]
}

2.3 Put an image to the input-output data container:

C++

ioData["image"] = imgCtx;

Python

ioData = {"image": imgCtx}

2.4 Call the faceDetector and pass the Context-container ioData that contains an image:

C++

faceDetector(ioData);

Python

faceDetector(ioData)

The result of calling faceDetector() will be appended to ioData container. The format of the output data is presented as a list of objects with the "objects" key. Each list object has the "class" key with the "face" value. The "bbox" (bounding box) key contains an array of four numbers of type double {x1, y1, x2, y2} that are relative coordinates of source image. The "confidence" key contains a confidence score of double number in a range of [0,1].

/*
{
 "objects": [{ "id": {"type": "long", "minimum": 0},
               "class": "face",
               "confidence": {"double",  "minimum": 0,  "maximum": 1},
               "bbox": [x1, y2, x2, y2] }]
}
*/

Follow iteration expression to get the result of face detection and draw it on a source image:

if (ioData.contains("objects"))
    for(const auto& obj : ioData["objects"])
    {
        if(obj["class"].getString().compare("face"))
            continue;
        const auto& rectCtx = obj.at("bbox");
        cv::Rect rect(cv::Point{rectCtx[0].getDouble()*input_image.cols, rectCtx[1].getDouble()*input_image.rows},
                      cv::Point{rectCtx[2].getDouble()*input_image.cols, rectCtx[3].getDouble()*input_image.rows});
        cv::rectangle(image, rect, {0, 255, 0}, 1);
    }

cv::imshow("img", image);
cv::waitKey();

note

Examples of using Face Detector:

• C++ - examples/cpp/processing_block/processing_block.cpp
• Python - examples/python/processing_blocks/block_name.py

3. GPU Acceleration

Face Detector can be used with GPU acceleration (CUDA). For more information, please follow this link.

Body Detection (C++/Python)

Requirements

• Windows x86 64-bit or Linux x86 64-bit system.
• Installed Face SDK package windows_x86_64 or linux_x86_64 (see Getting Started).

1. Creating a Human Body Detector

1.1 To create a Human Body Detector, follow steps 1-3 described in Creating a Processing Block and specify the values:

• "HUMAN_BODY_DETECTOR" for the "unit_type" key;
• path to Human Body Detector model file for the "model_path" key.

C++

configCtx["unit_type"] = "HUMAN_BODY_DETECTOR";

// default path to Human Body Detector model file - "share/bodydetectors/body.enc" in the Face SDK's root directory
configCtx["model_path"] = "share/bodydetectors/body.enc";

// optional, default value is 0.5
configCtx["confidence_threshold"] = 0.5;

// optional, default value is 0.5
configCtx["iou_threshold"] = 0.5;

Python

configCtx = {
    "unit_type": "HUMAN_BODY_DETECTOR",
    # the path is relative to the Face SDK root directory
    "model_path": "share/bodydetectors/body.enc",

    # optional, default values are specified after ":"
    "confidence_threshold": 0.5,
    "iou_threshold": 0.5
}

1.2 Create a Human Body Detector Processing block:

C++

pbio::ProcessingBlock bodyDetector = service->createProcessingBlock(configCtx);

Python

bodyDetector = service.create_processing_block(configCtx)

2. Body Detection

2.1 Create a Context container ioData for input-output data using the createContext() method:

auto ioData = service->createContext();

2.2 Create a Context container imgCtx with RGB-image following the steps described on Creating a Context container with RGB-image.

# copy an image into the binary format
input_rawimg = image.tobytes()
# put an image into the container
imageCtx = {
    "blob": input_rawimg,
    "dtype": "uint8_t",
    "format": "NDARRAY",
    "shape": [dim for dim in img.shape]
}

2.3 Put an image to the input-output data container:

C++

ioData["image"] = imgCtx;

Python

ioData = {"image": imgCtx}

2.4 Call the bodyDetector and pass the Context-container ioData that contains an image:

C++

bodyDetector(ioData);

Python

bodyDetector(ioData)

The result of calling bodyDetector() will be appended to ioData container. The format of the output data is presented as a list of objects with the "objects" key. Each list object has the "class" key with the "body" value. The "bbox" (bounding box) key contains an array of four numbers of type double {x1, y1, x2, y2} that are relative coordinates of source image. The "confidence" key contains a confidence score of double number in a range of [0,1].

/*
{
 "objects": [{ "id": {"type": "long", "minimum": 0},
               "class": "body",
               "confidence": {"double",  "minimum": 0,  "maximum": 1},
               "bbox": [x1, y2, x2, y2] }]
}
*/

Follow iteration expression to get the result of body detection and draw it on a source image:

if (ioData.contains("objects"))
    for(const auto& obj : ioData["objects"])
    {
        if(obj["class"].getString().compare("body"))
            continue;
        const auto& rectCtx = obj.at("bbox");
        cv::Rect rect(cv::Point{rectCtx[0].getDouble()*input_image.cols, rectCtx[1].getDouble()*input_image.rows},
                      cv::Point{rectCtx[2].getDouble()*input_image.cols, rectCtx[3].getDouble()*input_image.rows});
        cv::rectangle(image, rect, {0, 255, 0}, 1);
    }

cv::imshow("img", image);
cv::waitKey();

note

Examples of using Human Body Detector:

• C++ - examples/cpp/processing_block/processing_block.cpp
• Python - examples/python/processing_blocks/body_detector.py

3. GPU Acceleration

Human Body Detector can be used with GPU acceleration (CUDA). For more information, please follow this link.

Object Detection (C++/Python)

Requirements

• Windows x86 64-bit or Linux x86 64-bit system.
• Installed Face SDK package windows_x86_64 or linux_x86_64 (see Getting Started).

1. Creating an Object Detector

1.1 To create an Object Detector, follow steps 1-3 described in Creating a Processing Block and specify the values:

• "OBJECT_DETECTOR" for the "unit_type" key;
• path to Object Detector model file for the "model_path" key.

C++

configCtx["unit_type"] = "OBJECT_DETECTOR";

// default path to Object Detector model file - "share/objectdetectors/det-object.enc" in the Face SDK's root directory
configCtx["model_path"] = "share/objectdetectors/det-object.enc";

// optional, default value is 0.5
configCtx["confidence_threshold"] = 0.5;

// optional, default value is 0.5
configCtx["iou_threshold"] = 0.5;

Python

configCtx = {
    "unit_type": "OBJECT_DETECTOR",
    # the path is relative to the Face SDK root directory
    "model_path": "share/objectdetectors/det-object.enc",

    # optional, default values are specified after ":"
    "confidence_threshold": 0.5,
    "iou_threshold": 0.5
}

1.2 Create an Object Detector:

C++

pbio::ProcessingBlock objectDetector = service->createProcessingBlock(configCtx);

Python

objectDetector = service.create_processing_block(configCtx)

2. Object Detection

2.1 Create a Context container ioData for input-output data using the createContext() method:

auto ioData = service->createContext();

2.2 Create a Context container imgCtx with RGB-image following the steps described on Creating a Context container with RGB-image.

# copy an image into the binary format
input_rawimg = image.tobytes()
# put an image into the container
imageCtx = {
    "blob": input_rawimg,
    "dtype": "uint8_t",
    "format": "NDARRAY",
    "shape": [dim for dim in img.shape]
}

2.3 Put an image to the input-output data container:

C++

ioData["image"] = imgCtx;

Python

ioData = {"image": imgCtx}

2.4 Call objectDetector and pass a Context-container ioData that contains an image:

C++

objectDetector(ioData);

Python

objectDetector(ioData)

The result of calling objectDetector() will be appended to ioData container. The format of the output data is presented as a list of objects with the "objects" key. Each list object has the "class" key with the value according to detected object. The "bbox" (bounding box) key contains an array of four numbers of type double {x1, y1, x2, y2} that are relative coordinates of source image. The "confidence" key contains a confidence score of double number in a range of [0,1].

/*
{
 "objects": [{ "id": {"type": "long", "minimum": 0},
               "class": <class_of_detected_object>,
               "confidence": {"double",  "minimum": 0,  "maximum": 1},
               "bbox": [x1, y2, x2, y2] }]
}
*/

Follow iteration expression to get the result of object detection and draw it on a source image:

if (ioData.contains("objects"))
    for(const auto& obj : ioData["objects"])
    {
        std::string class_name = obj.at("class").getString();
        const auto& rectCtx = obj.at("bbox");
        cv::Rect rect(cv::Point{static_cast<int>(rectCtx[0].getDouble()*image.size[1]), static_cast<int>(rectCtx[1].getDouble()*image.size[0])},
                      cv::Point{static_cast<int>(rectCtx[2].getDouble()*image.size[1]), static_cast<int>(rectCtx[3].getDouble()*image.size[0])});
        cv::rectangle(image, rect, {0, 255, 0}, 1);
    }

cv::imshow("img", image);
cv::waitKey();

note

Examples of using Object Detector:

• C++ - examples/cpp/processing_block/processing_block.cpp
• Python - examples/python/processing_blocks/objects_detector.py

3. GPU Acceleration

Object Detector can be used with GPU acceleration (CUDA). For more information, please follow this link.