Reactive task planning for robotic sequential manipulation of rigid/soft objects

Abstract

The recent emergence of Industry 5.0 demands increased efforts towards intel­ligent, human-machine collaborative, sustainable, resilient production, requiring the system to implement varying levels of autonomy in all forms of human-robot interaction (HRI). Motivated by this manufacturing demand, the development of sequential manipulation tasks under both fully automatic mode and human-aware mode is an important research problem. To develop efficient solutions to automate complex robotic tasks, this work takes the packing of long linear elastic objects into common-size boxes as a case of study, where a new action planning approach for sequential manipulation is proposed. To achieve this, a hybrid geometric model is developed to handle large-scale occlusions combining an online vision-based method and an offline reference template. Then, an optimal packing strategy is designed to properly select a shape-box pair from a library of prior knowledge. Next, a reference point gen­erator is introduced to automatically plan the target poses for the pre-designed action primitives. Finally, an action planner integrates these components enabling the execution of high-level behaviors and the accomplishment of complex packing manipulation tasks. To validate the proposed approach, a detailed experimental study is conducted with multiple types and lengths of objects and packing boxes. To develop effective solutions to switch between fixed automated processes and human-aware modes, this work introduces a novel strategy that enables humans and robots to share spaces while collaboratively performing a manufacturing pro­cess. The proposed method seamlessly transitions between temporary HRI (i.e., human-aware mode) and long-horizon automated tasks (i.e., fully automatic mode) to establish resilient and energy-efficient production systems. To achieve this, a task progress monitor is developed that enables the decomposition of a complex manipulation task into several robot-centric action sequences, which are then fur­ther divided into a series of three-phase subtasks. A trigger signal switches modes based on the detected human action and its contribution to the task. A human agent coefficient matrix is computed using selected environmental features to pro­vide an appropriate cut-point for each robot to reactively execute the manipulation actions. To evaluate the performance of the proposed method, an extensive exper­imental study is conducted with robotic manipulators performing representative manufacturing tasks in both automatic mode and collaboration with humans. The reported body of work in this thesis contributes to the field of HRI with the potential to enhance sustainability in the context of Industry 5.0, which paves the way for intelligent manufacturing processes of future societies.