Introduction

This example shows how to write an OpenVx application which integrates the camera pipeline to TIDL based Object Detection pipeline. The demo is validated on Pelee network available at, https://github.com/Robert-JunWang/Pelee. Raw pixels from sensor is captured via CSI2Rx and sent to VISS for image processing. The resulting image is written in NV12 format which is then sent to LDC for fish-eye rectification. The rectified image is scaled using MSC to produce a DL resolution of 1024x512 as required by the Pelee network. As a pre-processing step, the scaled NV12 resolution is converted to RGB format and padded as required on C66x_1 DSP. The pre-processed input is given to TIDL running on C7x DSP for inference. As a post-processing step, the TIDL output is read and top detections are rendered as 2D boxes on the provided image on C66x_2 DSP. The final output is blended with system performance statistics and displayed on a 1920x1080 screen.

Supported plaforms

Platform	Linux x86_64	Linux+RTOS mode	QNX+RTOS mode	SoC
Support	NO	YES	YES	J721e / J721S2 / J784S4

Data flow

Steps to run the application on J7 EVM

Build the application and related libraries as mentioned in Build Instructions for Linux+RTOS mode
A sample "app_od_cam.cfg" for Object detection is provided under "/opt/vision_apps/" on the rootfs partition.
Create a folder and keep all the .yuv files used for detection, there are a few images under "/opt/vision_apps/test_data/psdkra/tidl_demo_images".
In case a different network is chosen, update the dl_size accordingly for network input resolution.
Run the app as shown below
./run_app_tidl_od_cam.sh
Select a sensor from a list of supported sensors
Choose if LDC should be enabled or not
Select the number of cameras to run the network

Steps to run the application on PC Linux x86_64

This application cannot be run on PC

Sample Output

Shown below is a example for running object detection on 4 IMX390 2MP cameras