Multi-granularity man-machine co-fusion environment perception method based on deep learning

Abstract

The invention discloses a multi-granularity man-machine co-fusion environment perception method based on deep learning. The method comprises the following steps: acquiring an RGB image and a depth image of a man-machine co-fusion scene; inputting the RGB image and the depth image into a coding network to obtain a coded image; inputting the coded image into a pyramid pooling module to obtain a pooling image; inputting the pooling image into a decoding network to obtain a decoded image; inputting the decoded image into a multi-granularity segmentation output module to obtain scene segmentation images of different granularity levels; wherein the granularity level comprises a region level, an entity level and a partial level of the entity. The scene segmentation images of different granularity levels provide more perfect environment perception capability for the collaborative robot, so that the collaborative robot can adaptively switch environment perception segmentation results of different granularities according to different environments and tasks, and subsequent collaborative behavior decision and motion planning can be better carried out. 本发明公开了一种基于深度学习的多颗粒度人机共融环境感知方法，获取人机共融场景的RGB图像和深度图像；将所述RGB图像和所述深度图像输入编码网络，得到编码图像；将所述编码图像输入金字塔池化模块，得到池化图像；将所述池化图像输入解码网络，得到解码图像；将所述解码图像输入多粒度分割输出模块，得到不同粒度等级的场景分割图像；其中，所述粒度等级包括区域等级、实体等级以及实体的部分等级。不同粒度等级的场景分割图像为协作机器人提供更加完善的环境感知能力，使其能够根据环境和任务的不同自适应地切换不同粒度的环境感知分割结果，从而更好地进行后续协作行为决策和运动规划。