dnn_image_classification

Classify the images with a DNN model, e.g. PP-ResNet50 (ImageNet).

Image classification, by default with PP-ResNet50 (ImageNet 1000 classes) of opencv_zoo, or with a model of the same input (RGB, normalized by the mean and the standard deviation of ImageNet) and output (1 x classes), e.g. MobileNet of opencv_zoo. ~detections has the top_k classes in the order of the score, whose rect is the image.

opencv_apps	3.0.0 and later
ROS	ROS One, jazzy, kilted, lyrical and later
OpenCV	OpenCV 4.5.4 or later
Group	Image classification, the nodes of which publish ~/detections (DetectionArrayStamped), the classes with their scores

Usage

# ROS 2
ros2 launch opencv_apps dnn_image_classification.launch.xml image:=/camera/image_raw debug_view:=false
# ROS 1
roslaunch opencv_apps dnn_image_classification.launch image:=/camera/image_raw debug_view:=false

Try it with the demo video

Download the input video of the demo above, publish it as image with image_publisher, and run the node on it, in the terminals:

wget https://ros-perception.github.io/opencv_apps/media/clips/tos_bridge.mp4
# terminal 1: publish the video as image in a loop
ros2 run image_publisher image_publisher_node tos_bridge.mp4 --ros-args -p publish_rate:=10.0 -r image_raw:=image
# terminal 2: run the node
ros2 launch opencv_apps dnn_image_classification.launch.xml image:=image
# terminal 3: view the result
ros2 run rqt_image_view rqt_image_view /dnn_image_classification/image

In ROS 1:

# terminal 1
rosrun image_publisher image_publisher tos_bridge.mp4 _publish_rate:=10 __name:=camera
# terminal 2
roslaunch opencv_apps dnn_image_classification.launch image:=camera/image_raw

See Credits of the demo videos for the license of the video.

Models

The node runs the classification models of ImageNet with the same input (224x224, RGB normalized by the mean and the standard deviation of ImageNet) and output (the logits of the 1000 classes), given by model. ImageNet has no class of the people, so the classes of this clip are the things around them.

PP-ResNet50 (default)

ros2 launch opencv_apps dnn_image_classification.launch.xml image:=image

The most accurate (top-1 82.3 % on ImageNet, 98 MB): use it unless the computer is slow, e.g. with a GPU or a desktop CPU.

MobileNetV1

ros2 launch opencv_apps dnn_image_classification.launch.xml image:=image model:=https://github.com/opencv/opencv_zoo/raw/48b10fa505aeb51fbd127fc60a5c660040e823b7/models/image_classification_mobilenet/image_classification_mobilenetv1_2022apr.onnx

MobileNetV1 of opencv_zoo (top-1 67.6 %, 16 MB, OpenCV 4.9 or later): much faster and smaller than PP-ResNet50, for a small computer without a GPU, e.g. a Raspberry Pi; MobileNetV2 is a little more accurate.

MobileNetV2

ros2 launch opencv_apps dnn_image_classification.launch.xml image:=image model:=https://github.com/opencv/opencv_zoo/raw/48b10fa505aeb51fbd127fc60a5c660040e823b7/models/image_classification_mobilenet/image_classification_mobilenetv2_2022apr.onnx

MobileNetV2 of opencv_zoo (top-1 69.4 %, 14 MB, OpenCV 4.9 or later): the more accurate of the two MobileNets at about the same speed, the first choice for a small computer.

Subscribed topics

Topic	Type
image	sensor_msgs/msg/Image

Published topics

Topic	Type
~/detections	opencv_apps/msg/DetectionArrayStamped
~/image	sensor_msgs/msg/Image

Parameters

See also the common parameters.

Name	Type	Default (range)	Changed at run time	Description
backend	string	“auto”	yes	Backend of cv::dnn: auto, cpu, opencl, opencl_fp16, cuda or cuda_fp16. auto uses CUDA when OpenCV is built with CUDA and a CUDA device is found, and the CPU otherwise.
classes	string	“”	yes	Path or URL of the class names of the model, one class per line, or empty for the 1000 classes of ImageNet
input_height	int	224 (1 - 4096)	yes	Height of the network input
input_width	int	224 (1 - 4096)	yes	Width of the network input
model	string	“”	yes	Path or URL of the model (onnx), empty for PP-ResNet50 of opencv_zoo, e.g. MobileNet of opencv_zoo (OpenCV 4.9 or later). The URL is downloaded to ~/.ros/opencv_apps/models/ at the first time
queue_size	int	3	no, restart the node	Specify queue_size of input image subscribers
softmax	bool	true	yes	Apply softmax to the output of the model to get the probabilities, true for the models whose output is the logits (e.g. PP-ResNet50 and MobileNet of opencv_zoo), false for the models whose output is the probabilities
top_k	int	5 (1 - 1000)	yes	Number of the classes to publish

Source

src/nodelet/dnn_image_classification_nodelet.cpp, launch/dnn_image_classification.launch.xml