An efficient continual learning grasp detection method with lightweight RGB-D fusion and automated data annotation for kitchen waste sorting

Abstract

Utilizing robots for sorting and impurity removal of kitchen waste presents a challenging task. When performing such tasks, visual servo systems often require a mechanism to continuously accommodate the new tasks and environments. First, we propose a lightweight RGB-D fusion module, which efficiently fuses RGB and depth information at the channel level, achieving an over 2.1% improvement in grasping accuracy compared to traditional RGB-D fusion methods without increasing computational burden. Second, we introduce a labels automated annotation module based on detecting the degree of gripper closure. This module determines grasp success or failure by assessing the actual closure of the gripper. Finally, through these labels and manual correction, we constructed five grasping tasks to build a new continual learning dataset [kitchen waste grasp on continual learning (KWG-CL)] and proposed a novel continual learning method. By using this method, our algorithm achieves the highest average image-level grasp accuracy (77.28%) on the KWG-CL compared to state-of-the-art approaches, while exhibiting the least catastrophic forgetting. In addition, we also validated this in a real-world robot system. To the best of the authors' knowledge, this is the first application of the continual learning method in kitchen waste sorting.