Skip to main content
Version: 3.15.0

Body Detection

Human Body Detector​

The Human Body Detector is a component used to detect human bodies on an image. The detection result is a bounding rectangle (a frame) around the detected body.

At this page, you'll learn how to run the Human Body Detection sample on Windows or Linux and how to integrate Human Body Detector component in your C++ project.

Human Body Detection Sample​

The body_detector sample demonstrates human bodies detection on an image.

note

To run the body_detector sample you need to install Face SDK 3.14 or higher with Windows or Linux platform files (according to your operating system).

  • If Face SDK is not installed, follow the installation instructions Getting Started. Select platform in the "Selection Components" section.

  • If Face SDK is installed without Windows or Linux platform files, use the utility maintenancetool and install platform by selecting it in the "Selection Components" section.

Startup arguments:

  • Path to the input image file
  • Path to the body.enc file
note

Default path to Human Body Detector model file is share/bodydetectors/body.enc in the Face SDK's root directory

Console commands to launch body_detector from the bin directory:

  • Linux: ./body_detector group/01.jpg ../share/bodydetectors/body.enc
  • Windows: .\body_detector.exe group\01.jpg ..\share\bodydetectors\body.enc

The results of body detection will be displayed in the window.

Result of body detection

Error messages, if any, are also printed in the console.

Troubleshooting

  • Error Assertion failed (ERROR: not found libonnxruntime.so), error code: 0x032ad038:

    • Check existence of the file lib/libonnxruntime.so in the Face SDK's root directory
    • Add path to the file lib/libonnxruntime.so in environment variable LD_LIBRARY_PATH

  • Error Assertion failed (ERROR: not found onnxruntime.dll), error code: 0x032ad038:

    • Check existence of the file bin/onnxruntime.dll in the Face SDK's root directory

note

Source code of the body_detector sample available at the link. Also, the source code and instructions of building (file README.txt) are available in the examples/cpp/body_detector_cpp/ directory of the Face SDK distributive.

Body Detection (C++)​

In this section you'll learn how to integrate Human Body Detector component to your C++ project.

Context​

Context is a heterogeneous container that consists of a set of hierarchically organized data presented in the form of key–value pairs. The closest analogue of Context is a JSON object. Each Context object can contain a scalar object (integer, real, boolean, string), a memory area or pointer, an array of Context objects, or an associative container of string-Context pairs, with unlimited nesting.

Example of using a Context object
Context array_elem0;
array_elem0["name"] = "Julius Zeleny";
array_elem0["phone"] = 11111111111l;
array_elem0["social_score"] = 0.999;
array_elem0["verified"] = true;

std::cout << array_elem0["name"].getString() << std::endl;      // "Julius Zeleny"
std::cout << array_elem0["phone"].getLong() << std::endl;       // 11111111111l
std::cout << array_elem0["social_score"].getDouble() << std::endl;// 0.999
std::cout << array_elem0["verified"].getBool() << std::endl;        // 1

Context array_elem1;
array_elem1["name"] = "Owen Cohron";
array_elem1["phone"] = 22222222222;
array_elem1["social_score"] = 0.999;
array_elem1["verified"] = false;

Context array_elem2;
array_elem2["name"] = std::string("Sarah Connor");
array_elem2["phone"] = 333333333l;
array_elem2["social_score"] = 0.5;
array_elem2["verified"] = true;

std::cout << array_elem2["name"].getString() << std::endl;  // "Sarah Connor"

Context array;
array.push_back(array_elem0);
array.push_back(array_elem1);
array.push_back(array_elem2);

std::cout << array[0]["phone"].getLong() << std::endl;      // 11111111111l
std::cout << array[1]["phone"].getLong() << std::endl;      // 22222222222
std::cout << array[2]["phone"].getLong() << std::endl;      // 333333333l

size_t array_sz = array.size();
std::cout << array_sz << std::endl; // 3

for(size_t i = 0; i < array_sz; ++i) {
array[i]["phone"];
}

Context full;
full["friends"] = array;

std::cout << full["friends"][1]["social_score"].getDouble() << std::endl; // 0.999

// copy cv::Mat to Context
int width{640}, height{480}, channels{3};
cv::Mat img = cv::imread("photo.jpg");
full["img"]["dtype"] = "uint8";
full["img"]["shape"].push_back(width);
full["img"]["shape"].push_back(height);
full["img"]["shape"].push_back(channels);
full["img"]["ptr"] = img.data;
full["img"]["data"].setDataPtr(img.data, img.total()*img.elemSize());

// create cv::Mat from Context
unsigned char* img_buf = full["img"]["data"].getDataPtr();
cv::Mat imgCopy = cv::Mat(height, width, CV_8UC3, img_buf);

1. Creating a Human Body Detector​

1.1. Create a FacerecService.

1.2. Then create a Context container:

auto modelCtx = service->createContext();

1.3. Define fields in the created context container for creating a Processing block:

// mandatory
modelCtx["unit_type"] = "HUMAN_BODY_DETECTOR";

// mandatory
// default path to Human Body Detector model file is share/bodydetectors/body.enc in the Face SDK's root directory
modelCtx["model_path"] = "share/bodydetectors/body.enc";

// default value is 0.5
modelCtx["confidence_threshold"] = 0.2;

// default value is 0.5
modelCtx["iou_threshold"] = 0.4;

// default location of the onnxruntime library in Face SDK distributive: the "lib" folder for Linux platfrom or the "bin" folder for Windows platfrom
// you can specify your own path to onnxruntime library
// if value is not specified, the os-specific default search order will be used
modelCtx["ONNXRuntime"]["library_path"] = "../lib"

1.4. Create a Human Body Detector Processing block:

pbio::ProcessingBlock bodyDetector = service->createProcessingBlock(modelCtx);

2. Body Detection​

The body detector takes a Context container including an image in binary format:

/*
{
 "image" : { "format": "NDARRAY",
             "blob": <blob_value>,
             "dtype": "uint8_t",
             "shape": [] }
}
*/

The "blob" key contains a pointer to data. The pointer is set by the function void Context::setDataPtr(void* ptr, int copy_sz), where copy_sz is the size of memory in Bytes, that will be copied, and then automatically released. Copying will not perform if 0 is passed as argument copy_sz.

The "dtype" can contain one of these values: "uint8_t", "int8_t", "uint16_t", "int16_t", "int32_t", "float", "double". This is according to OpenCV types: CV_8U, CV_8S, CV_16U, CV_16S, CV_32S, CV_32F, CV_64F.

2.1. Create a Context container for image with using the createContext() method:

auto imgCtx = service->createContext();

2.2. Get a cv::Mat of RGB-image:

cv::Mat image = cv::imread(input_image_path, cv::IMREAD_COLOR);
cv::Mat input_image;
cv::cvtColor(image, input_image, cv::COLOR_BGR2RGB);

2.3. Convert an image from cv::Mat to binary format and put it to Context container.

void cvMatToBSM(pbio::Context& bsmCtx, const cv::Mat img, bool copy)
{
    bsmCtx["format"] = "NDARRAY";
    size_t copy_sz = copy ? img.total()*img.elemSize() : 0;
    bsmCtx["blob"].setDataPtr(img.data, copy_sz);
    bsmCtx["dtype"] = CvTypeToStr.at(img.depth());
    for(int i = 0; i < img.dims; ++i)
        bsmCtx["shape"].push_back(img.size[i]);
    bsmCtx["shape"].push_back(img.channels());
}

cvMatToBSM(imgCtx, input_image);

2.4. Create a Context container:

auto ioData = service->createContext();
ioData["image"] = imgCtx;

2.5. Call the bodyDetector and pass the ioData:

bodyDetector(ioData);

The result of calling bodyDetector() will be appended to ioData container. The format of the output data is presented as list of objects with the "objects" key. Each list object has the "class" key with the "body" value. The "bbox" (bounding box) key contains an array of 4th numbers of type long {x1, y1, x2, y2} that are coordinates in pixels of source image. The "score" key contains a float number in a range of [0,1].

/*
{
 "objects": [{ "id": <id_value>,
               "class": "body",
               "score": <score_value>,
               "bbox": [] }]
}
*/

The Context objects allow iterating by its values, so you can use following iteration expression to get the result of bodies detection and draw it on a source image:

for(const auto& obj : ioData["objects"])
{
    if(obj["class"].getString().compare("body"))
        continue;
    const auto& rectCtx = obj.at("bbox");
    cv::Rect rect(cv::Point{rectCtx[0].getLong(), rectCtx[1].getLong()},
                  cv::Point{rectCtx[2].getLong(), rectCtx[3].getLong()});
    cv::rectangle(img, rect, {0, 255, 0}, 1);
}

cv::imshow("img", img);
cv::waitKey();

3. GPU Acceleration​

Human Body Detector can be used with GPU acceleration (CUDA). To activate acceleration you need to define the ["use_cuda"] key with the true value for Processing Block container (see Creating a Human Body Detector).