Realtime Semantic Segmentation on Jetson Nano in Python and C++

Recently I updated the Hello AI World project on GitHub with new semantic segmentation models based on FCN-ResNet18 that run in realtime on Jetson Nano, in addition to Python bindings and examples for segmenting images and live camera video. Like with the other networks in the project, they use NVIDIA TensorRT underneath for optimized performance.

The segmentation models have been trained in PyTorch at various resolutions on datasets including Cityscapes, DeepScene, Multi-Human Parsing, Pascal VOC, and SUN RGB-D:

Below is example Python code that segments an image using the library:

import jetson.inference
import jetson.utils

# load an image (into shared CPU/GPU memory)
img, width, height = jetson.utils.loadImageRGBA("my_image.jpg")

# allocate output image for the overlay/mask (float4)
img_output = jetson.utils.cudaAllocMapped(width * height * 16)

# load the segmentation model
net = jetson.inference.segNet("fcn-resnet18-voc-512x320")

# process the segmentation network
net.Process(img, width, height)

# perform the visualization
net.Overlay(img_output, width, height)
	
# save the image to disk

jetson.utils.saveImageRGBA("my_output.jpg", img_output, width, height)

See the updated documentation for the new tutorial steps around semantic segmentation. Samples and APIs are provided for both Python and C++ depending on your preference.

Grab the latest code and models from here...https://github.com/dusty-nv/jetson-inference