Interaction and control in robotics : from assistance to collaborative manipulation

Abstract

Throughout human history, we have collaborated with one another to accomplish more challenging tasks together. With technological advancements, humans have invented a myriad of machines and robots to assist in our daily lives and tasks. Nowadays, we are accustomed to working alongside robots, such as adopting robotic arms to move large metal pieces in car manufacturing. Beyond using robots to assist humans, researchers have also focused on en­abling robots to collaborate and work with one another, introducing multi-robot systems. To further enhance the intelligence of robots and minimize the differ­ences between human and machine, researchers have also developed robots that can understand and use human tools to manipulate objects while carrying out tasks. This thesis focuses on the interactions and controls in robotics, exploring vari­ous aspects from perception and capability sharing in multi-robot collaboration to collaborative manipulation through tool usage. To investigate potential applica­tion scenarios, the work includes three case studies: (1) Welder training assistant with augmented perception, (2) Capability sharing in heterogeneous multi-robot systems, and (3) Non-prehensile tool manipulation. In the welder training assistant project, a multi-sensor interface was developed to assist humans in teaching and learning arc welding more efficiently through performance visualization and quantification. For the multi-robot system, a dis­tributed ontological collaborative task allocation framework was proposed, fo­cusing on allocating tasks among robots based on their capabilities. In the tool manipulation project, a non-prehensile tool manipulation methodology was de­veloped, utilizing a Large Language Model for task decomposition. To enable fine tool motion correction with objects confined in a limited area, an incremental stepping manipulation approach was also designed. The proposed methodologies are validated and analyzed through extensive ex­periments to demonstrate the efficiency and effectiveness of the developed solu­tions in real-world scenarios.