Camera based Semantic Segmentation Application

Table of Contents

• Introduction
• Supported plaforms
• Data flow
• Steps to run the application on J7 EVM
• Steps to run the application on PC Linux x86_64
• Sample Output

Introduction

This example shows how to write an OpenVx application which integrates the camera pipeline to TIDL based Semantic Segmentation pipeline. The demo is validated on tiadsegNet_v2 network. Raw pixels from sensor is captured via CSI2Rx and sent to VISS for image processing. The resulting image is written in NV12 format which is then sent to LDC for fish-eye rectification. The rectified image is scaled using MSC to produce a DL resolution of 768x384 as required by the network. As a pre-processing step, the scaled NV12 resolution is converted to RGB format and padded as required on C66x_1 DSP. The pre-processed input is given to TIDL running on C7x DSP for inference. As a post-processing step, the TIDL output is read and each pixel is blended with a color map on the provided image on C66x_2 DSP. The final output is blended with system performance statistics and displayed on a 1920x1080 screen.

Supported plaforms

Platform	Linux x86_64	Linux+RTOS mode	QNX+RTOS mode	SoC
Support	NO	YES	YES	J721e / J721S2 / J784S4

Data flow

Steps to run the application on J7 EVM

1. Build the application and related libraries as mentioned in Build Instructions for Linux+RTOS mode
2. A sample "app_seg_cam.cfg" for Semantic Segmentation is provided under "/opt/vision_apps/" on the rootfs partition.
3. In case a different network is chosen, update the dl_size accordingly for network input resolution.
4. Run the app as shown below

   ./run_app_tidl_seg.sh

5. Select a sensor from a list of supported sensors
6. Choose if LDC should be enabled or not
7. Select the number of cameras to run the network

Steps to run the application on PC Linux x86_64

1. This application does not run on PC

Sample Output

Shown below is a example for running semantic segmentation on 4 IMX390 2MP cameras