Development of a Method for Detecting Cutting Points of Wilted Leaves Using 2D Image Keypoints and 3D Point Cloud

In agriculture, removing withered leaves is crucial for maintaining crop quality and preventing pests and diseases. This paper proposes a method for estimating the pose of wilted leaves and removing wilted leaves. While estimating precisely 3D location and pose information is essential for this task, most of previous methods for pose estimation have difficulty in processing time due to the complexity of feature extraction from 3D point cloud. The proposed method utilizes the YOLOv8 keypoint detection model to obtain image information, and YOLOv7 instance segmentation to isolate the region of interest. We calculate the 3D coordinates of the cutting point and the 3D rotation of the wilted leaf by applying RANSAC plane fitting and projection on 3D point cloud, simplifying the information required for the task. The method was validated using five types of model plant pots, achieving an average processing time of 98.584 ms and an accuracy of 3.83 mm. Additionally, the method recorded a 78% success rate in the removal of withered leaves.