Sophon SDK development
Sophon SDK is a customized deep learning SDK designed for model optimization and efficient neural network inference. It provides a full-stack solution for the development and deployment of deep learning applications.
Sophon SDK components
Core toolset
Tpu-nntc
Handles offline compilation and optimization of neural network models trained in third-party deep learning frameworks, generating the final binary BModel. Supported frameworks include Caffe, Darknet, MXNet, ONNX, PyTorch, PaddlePaddle, TensorFlow, and others.
Libsophon
Includes BMCV, BMRuntime, BMLib, and other libraries used for hardware management, such as VPP and TPU, to perform image processing, tensor operations, model inference, and other deep learning application tasks.
Sophon-mw
Wraps BM-OpenCV, BM-FFmpeg, and other libraries to manage VPU, JPU, and additional hardware resources. It also supports RTSP and GB28181 stream parsing, video encoding and decoding acceleration, and other operations, enabling deep learning application development.
Sophon-sail
Provides high-level interfaces for Python/C++ and encapsulates low-level library interfaces, including BMRuntime, BMCV, BMDecoder, BMLib, and others, to facilitate deep learning application development.
Advanced toolset
Tpu-mlir
Provides a complete toolchain for the TPU compiler project, enabling the conversion of pretrained neural networks from various frameworks into binary BModel for execution on Sophon TPU. Currently supported frameworks include TFLite, ONNX, and Caffe.
Tpu-perf
A comprehensive toolset for evaluating model performance and accuracy.
Tpu-kernel
A chip-level development interface that supports both specialized instructions for accelerating deep learning business logic and general-purpose instructions for optimizing various custom algorithms.
SDK directory structure
The Sophon SDK consists of eleven directories:
| Directory name | Description |
|---|---|
| libsophon | Libraries for hardware acceleration of Sophon devices, including image processing, tensor operations, model inference, and other tasks. |
| sophon-mw | Multimedia library supporting hardware acceleration for devices based on Sophon. |
| tpu-kernel | Interface for low-level development that provides access to instructions for accelerating deep learning logic and custom algorithms. |
| tpu-nntc | TPU compilation toolchain for optimizing models and converting them into a format supported by Sophon chips. |
| tpu-mlir | TPU compilation toolchain designed for converting trained neural networks into a binary format that operates on Sophon TPU. |
| tpu-perf | Toolset for evaluating the performance and accuracy of models. |
| sophon-pipeline | High-performance inference platform based on pipelines for executing deep learning tasks. |
| sophon-img | Setup for operation in SoC (System on Chip) mode for Sophon devices. |
| sophon-demo | Collection of examples demonstrating the use of the SDK for single models or scenarios. |
| sophon-sail | Library of interfaces that encapsulates low-level interfaces into an API for C++/Python. |
| sophon-rpc | Mechanism for offloading computational tasks from the PCIe card to the ARM CPU installed on the card. |
Directory structure of the unpacked SDK
1 SophonSDK
2
3├── libsophon_<date>_<hash>
4│ ├── BMCV_Technical_Reference_Manual.pdf
5│ ├── BMCV.pdf
6│ ├── BMLib_Technical_Reference_Manual.pdf
7│ ├── BMLIB.pdf
8│ ├── BMRuntime Technical Reference Manual.pdf
9│ ├── BMRUNTIME.pdf
10│ ├── libsophon_<x.y.z>_aarch64.tar.gz #Arm64 machine, libsophon installar for other LINUX systems
11│ ├── libsophon_<x.y.z>_x86_64.tar.gz #X86x64 machine, libsophon installar for other LINUX systems
12│ ├── libsophon_dockerfile
13│ ├── libsophon.MD5
14│ ├── LIBSOPHON_User_Guide.pdf
15│ ├── LIBSOPHON.pdf
16│ ├── release_version.txt
17│ ├── sophon-driver_<x.y.z>_amd64.deb #x86_64 machine，PCIe card driver installation file corresponding to Debian/Ubuntu system
18│ ├── sophon-driver_<x.y.z>_arm64.deb #arm64 machine，PCIe card driver installation file corresponding to Debian/Ubuntu system
19│ ├── sophon-libsophon_<x.y.z>_amd64.deb #x86_64 machine，libsophon runtime environment installation file corresponding to Debian/Ubuntu system
20│ ├── sophon-libsophon_<x.y.z>_arm64.deb #arm64 machine，libsophon runtime environment installation file corresponding to Debian/Ubuntu system
21│ ├── sophon-libsophon-dev_<x.y.z>_amd64.deb #x86_64 machine，libsophon runtime environment installation file corresponding to Debian/Ubuntu system
22│ └── sophon-libsophon-dev_<x.y.z>_arm64.deb #arm64 machine，libsophon runtime environment installation file corresponding to Debian/Ubuntu system
23├── sophon-demo_<date>_<hash>
24│ ├── release_version.txt
25│ ├── sophon-demo.MD5
26│ └── sophon-demo_v<x.y.z>_b909566_20221027.tar.gz #Comprehensive routines for a single model or scenario
27├── sophon-img_<date>_<hash>
28│ ├── bsp-debs
29│ ├── bsp_update.tgz
30│ ├── libsophon_soc_<x.y.z>_aarch64.tar.gz #Files required for cross-compilation
31│ ├── release_version.txt
32│ ├── sdcard.tgz #SoC card brush package
33│ ├── SOPHON_BSP_Technical_Reference_Manual.pdf
34│ ├── SOPHON BSP.pdf
35│ ├── sophon-img.MD5
36│ ├── sophon-soc-libsophon_<x.y.z>_arm64.deb #Libsophon installation files corresponding to SoC platform
37│ ├── sophon-soc-libsophon-dev_<x.y.z>_arm64.deb #Libsophon installation files corresponding to SoC platform
38│ ├── system.tgz
39│ └── tftp.tgz
40├── sophon-mw_<date>_<hash>
41│ ├── Multimedia FAQ.pdf
42│ ├── Multimedia Technical Reference Manual.pdf
43│ ├── Multimedia User Guide.pdf
44│ ├── MULTIMEDIA.pdf
45│ ├── MULTIMEDIA.pdf
46│ ├── MULTIMEDIA.pdf
47│ ├── release_version.txt
48│ ├── sophon-mw_<x.y.z>_aarch64.tar.gz #Arm64 machine, sophon-mw installar for other LINUX systems
49│ ├── sophon-mw_<x.y.z>_x86_64.tar.gz #Arm64 machine, sophon-mw installar for other LINUX systems
50│ ├── sophon-mw.MD5
51│ ├── sophon-mw-soc_<x.y.z>_aarch64.tar.gz
52│ ├── sophon-mw-soc-sophon-ffmpeg_<x.y.z>_arm64.deb #SoC platform, ffmpeg runtime environment installation files
53│ ├── sophon-mw-soc-sophon-ffmpeg-dev_<x.y.z>_arm64.deb #SoC platform, ffmpeg development environment installation files
54│ ├── sophon-mw-soc-sophon-opencv_<x.y.z>_arm64.deb #SoC platform, opencv runtime environment installation files
55│ ├── sophon-mw-soc-sophon-opencv-dev_<x.y.z>_arm64.deb #SoC platform, opencv development environment installation files
56│ ├── sophon-mw-soc-sophon-sample_<x.y.z>_arm64.deb #SoC platform, multimedia program sample file
57│ ├── sophon-mw-sophon-ffmpeg_<x.y.z>_amd64.deb #X86x64 machine, installation files for ffmpeg runtime environment corresponding to Debian/Ubuntu system
58│ ├── sophon-mw-sophon-ffmpeg_<x.y.z>_arm64.deb #arm64 machine, installation files for ffmpeg runtime environment corresponding to Debian/Ubuntu system
59│ ├── sophon-mw-sophon-ffmpeg-dev_<x.y.z>_amd64.deb #x86_64 machine, ffmpeg development environment installation file corresponding to Debian/Ubuntu system
60│ ├── sophon-mw-sophon-ffmpeg-dev_<x.y.z>_arm64.deb #arm64 machine, ffmpeg development environment installation file corresponding to Debian/Ubuntu system
61│ ├── sophon-mw-sophon-opencv_<x.y.z>_amd64.deb #x86_64 machine, opencv runtime environment installation file corresponding to Debian/Ubuntu system
62│ ├── sophon-mw-sophon-opencv_<x.y.z>_arm64.deb #arm64 machine, opencv runtime environment installation file corresponding to Debian/Ubuntu system
63│ ├── sophon-mw-sophon-opencv-abi0_<x.y.z>_amd64.deb #X86x64 machine, installation files for opencv runtime environment corresponding to CenterOS system
64│ ├── sophon-mw-sophon-opencv-abi0_<x.y.z>_arm64.deb #arm64 machine, opencv runtime environment installation file corresponding to CenterOS system
65│ ├── sophon-mw-sophon-opencv-abi0-dev_<x.y.z>_amd64.deb #x86x64 machine, opencv development environment installation file corresponding to CenterOS system
66│ ├── sophon-mw-sophon-opencv-abi0-dev_<x.y.z>_arm64.deb #arm64 machine, opencv development environment installation file corresponding to CenterOS system
67│ ├── sophon-mw-sophon-opencv-dev_<x.y.z>_amd64.deb #x86x64 machine, opencv development environment installation file corresponding to Debian/Ubuntu system
68│ ├── sophon-mw-sophon-opencv-dev_<x.y.z>_arm64.deb #arm64 machine, opencv development environment installation file corresponding to Debian/Ubuntu system
69│ ├── sophon-mw-sophon-sample_<x.y.z>_amd64.deb #x86x64 machine, sample files for multimedia programs
70│ └── sophon-mw-sophon-sample_<x.y.z>_arm64.deb #arm64 machine, sample files of multimedia programs
71├── sophon-pipeline_<date>_<hash>
72│ ├── release_version.txt
73│ ├── sophon-pipeline.MD5
74│ └── sophon-pipeline_v<x.y.z>_586366b_20221027.tar.gz #High performance inference framework based on pipeline
75├── sophon-rpc_<date>_<hash>
76│ ├── release_version.txt
77│ ├── sophon-rpc_<x.y.z>_amd64.deb #x86x64 machine, sophon-rpc installation file corresponding to Debian/Ubuntu system
78│ ├── sophon-rpc_<x.y.z>_arm64.deb #arm64 machine, sophon-rpc installation file corresponding to Debian/Ubuntu system
79│ ├── sophon-rpc_<x.y.z>.tar.gz #Sophon-rpc installation packages corresponding to other systems
80│ ├── sophon-rpc.MD5
81│ └── sophon-rpc.pdf
82├── sophon-sail_<date>_<hash>
83│ ├── release_version.txt
84│ ├── Sophon_Inference_zh.pdf
85│ ├── sophon-sail_<x.y.z>.tar.gz #Advanced interface library encapsulating BMLib, BMDecoder, BMCV, BMRuntime
86│ └── sophon-sail.MD5
87├── Sophon SDK.docx
88├── tpu-kernel_<date>_<hash>
89│ ├── release_version.txt
90│ ├── tpu-kernel-1684x_v<x.y.z>-0d0e513e-221027.tar.gz #Custom operator development tool
91│ └── tpu-kernel.MD5
92├── tpu-mlir_<date>_<hash>
93│ ├── release_version.txt
94│ ├── tpu-mlir.MD5
95│ └── tpu-mlir_v<x.y.z>-<hash>-<date>.tar.gz #Compiler tool chain
96├── tpu-nntc_<date>_<hash>
97│ ├── release_version.txt
98│ ├── tpu-nntc.MD5
99│ └── tpu-nntc_v<x.y.z>-<hash>-<date>.tar.gz #Model compilation quantization tool chain
100└── tpu-perf_v<x.y.z>
101 ├── md5sum.txt
102 ├── tpu_perf-<x.y.z>-py3-none-manylinux2014_aarch64.whl
103 ├── tpu_perf-<x.y.z>-py3-none-manylinux2014_x86_64.whl
104 └── tpu-perf-<x.y.z>.tar.gz #Tools for performance analysis and accuracy verification of the model
Core SDK modules
Hardware driver and runtime LIBSOPHON
Contains libraries such as BMCV, BMRuntime, BMLib, and others used for managing hardware like VPP, TPU, and other devices, performing operations such as image processing, tensor operations, model inference, and other tasks.
Multimedia library SOPHON-MW
Supports BM-OpenCV and BM-FFmpeg for hardware acceleration on Sophon devices, parsing RTSP streams, GB28181 streams, video and image encoding and decoding.
Toolchain for compiling and quantizing TPU models TPU-NNTC
Supports the conversion of models from frameworks such as Caffe, TensorFlow, PyTorch, MXNet, Darknet, PaddlePaddle, ONNX, and others. Also supports model quantization: original model -> FP32 UModel -> INT8 UModel -> INT8 BModel, and provides an automatic calibration tool for quantization.
Tensor operations and image processing library BMCV
Performs color space conversions, scaling, affine and projective transformations, linear transformations, frame processing, JPEG encoding, BASE64 encoding, NMS, sorting, and feature matching.
Hardware management library BMLib
Basic interfaces for managing device descriptors, memory management, data processing, API submission, synchronization, activating A53, and more.
AI computation acceleration library SAIL
Supports high-level interfaces for Python/C++ and acts as a wrapper for library interfaces such as BMRuntime, BMCV, BMDecoder.
Custom operators development library BMLang
High-level C++ library for Sophon TPUs, independent of hardware specifics, requiring no knowledge of hardware architecture. Code is written using tensor data (bmlang::Tensor) and computational operations (bmlang::Operator), then used with bmlang::compile or bmlang::compile_with_check to generate a BModel executable by the TPU. Also supports using ARM-CPU in BM168X for computations unsupported by the TPU.
Parallel algorithm acceleration library TPUKernel
Low-level programming interface based on the atomic operations interface of the Sophon chip, requiring knowledge of hardware architecture and instruction set.
Model performance and accuracy evaluation tool TPUPerf
Provides performance analysis and accuracy verification of models.
For more detailed information, please refer to this link.