To achieve this functionality, three main modules are needed: visual input, environment perception, and motion control.

• Visual Input Module: Captures images of the real or simulated environment and forwards them to the environment perception module.
• Environment Perception Module: Determines the car’s position on the track and provides data to the motion control module.
• Motion Control Module: Computes motion commands based on position data and sends them to the car for actuation.

These modules can be refined and implemented using NodeHub — Horizon’s “Intelligent Robot Application Center” that offers open-source Nodes for rapid robot development. By connecting different Nodes, you can complete the implementation of these three modules.

Under the “Peripheral Adaptation” category in NodeHub, select the “MIPI Camera Driver” Node to implement the visual input module. This Node supports the GC4663 wide-angle camera, providing a larger field of view suitable for racing car line tracking.

Features:

• Supports multiple resolutions: 1920x1080, 640x480.
• Publishes topics: /hbmem_image, /image_raw, /camera_info.
• Different resolutions selectable via ros2 launch files.

For optimal performance, use the configuration file mipi_cam_640x480_nv12_hbmem.launch.py, which publishes 640x480 NV12 images through shared memory.

Deployment Steps:

sudo apt update
sudo apt install -y tros-mipi-cam

Run Command:

source /opt/tros/setup.bash
ros2 launch mipi_cam mipi_cam_640x480_nv12_hbmem.launch.py

The Track Detection Node (in “Competition Zone”) detects the guiding lines’ position relative to the car using deep learning and publishes the midpoint of the track.

Features:

• Subscribes to /hbmem_image (hbm_img_msgs/msg/HbmMsg1080P), consistent with the visual input module’s output.
• Publishes to /racing_track_center_detection (geometry_msgs/msg/PointStamped).
• Model file path: /opt/nodehub_model/race_detection/race_track_detection.bin (replaceable).

Deployment Steps:

sudo apt update
sudo apt install -y tros-racing-track-detection-resnet

Run Command:

source /opt/tros/setup.bash
ros2 launch racing_track_detection_resnet racing_track_detection_resnet.launch.py

For training methods, refer to “Track Detection Model Training and Deployment Complete Explanation.”

The Car Line Following Control Node (under “Competition Zone”) receives messages from the Track Detection Node to control the car’s line-following behavior.

Features:

• Subscribes to /racing_track_center_detection (geometry_msgs/msg/PointStamped).
• Publishes control data to /cmd_vel (geometry_msgs/msg/Twist).

Deployment Steps:

sudo apt update
sudo apt install -y tros-racing-control

Run Command:

source /opt/tros/local_setup.bash
ros2 launch racing_control racing_control.launch.py \
  avoid_angular_ratio:=0.2 avoid_linear_speed:=0.1 \
  follow_angular_ratio:=-1.0 follow_linear_speed:=0.1

Under “Peripheral Adaptation,” select the Originbot Chassis Driver Node to receive control messages and manage motor movement.

Features:

• Subscribes to /cmd_vel (geometry_msgs/msg/Twist), consistent with the Motion Control Node.

Deployment Steps:

sudo apt update
sudo apt install -y tros-originbot-base tros-serial tros-originbot-msgs

Run Command:

source /opt/tros/setup.bash
ros2 launch originbot_base robot.launch.py

At this point, the visual line tracking demo setup is complete. The system integrates visual input, environment perception, and motion control modules to achieve full autonomous track following.