Advancements in unmanned aerial vehicle path planning : deep reinforcement learning approach for enhanced navigation

Abstract

The main objective of the thesis is to demonstrate a comprehensive study of the autonomous navigation of Unmanned Aerial Vehicles (UAVs) through intricate environments characterized by narrow gaps. Employing advanced Deep Reinforcement Learning (DRL) methodologies, this research introduces two innovative algorithms optimized for real-time UAV path planning. The first algorithm enhances the standard Deep Q-Network (DQN) by integrating a series of improvements that increase its adaptability and decision-making capabilities in complex environments. This enhancement involves the incorporation of a more efficient reward structure and a refined state-action space representation, allowing the UAV to autonomously generate optimized navigation paths. The enhanced DQN framework facilitates rapid adaptation to environmental variations, improving both the learning speed and robustness of the UAV's path planning. This results in more effective navigation, especially in environments with narrow gaps and dynamic obstacles. A key feature of this enhanced algorithm is its ability to map action commands directly from sensor data, thereby improving the UAV's real-time decision-making. Furthermore, by implementing a direction reward function, the algorithm incentivizes the UAV to optimize its trajectory towards target goals while penalizing deviations from the desired path. This approach strengthens the UAV's generalization ability, allowing it to perform effectively across a range of diverse operational scenarios. In parallel, this thesis addresses the complex challenge of autonomous navigation for UAVs in real environments. The study employs a sophisticated DRL approach using the Soft Actor-Critic (SAC) algorithm, which is specifically optimized for UAV path planning within a continuous action space. This method utilizes environmental image data to refine the accuracy of flight maneuvers and enhance obstacle avoidance capabilities. The efficacy of our approach has been substantiated through comprehensive simulations in Gazebo and empirical field tests, which demonstrate the algorithm's capability to enable UAVs to adeptly navigate through obstacles using depth maps. Furthermore, the study assesses the robustness of the SAC algorithm by juxtaposing it with conventional DRL methods, highlighting its superior performance in practical applications. This research makes a significant contribution to the advancement of UAV technology, particularly in autonomous motion planning, by incorporating advanced machine learning techniques. The findings and methodologies are accessible via the provided video link: https://www.youtube.com/watch?v=Nd_aMzejNXY. In general, this research advances UAV technology by integrating cutting-edge machine learning techniques into autonomous motion planning. It enhances the adaptability and efficacy of UAV navigation in narrow-gap environments and contributes significantly to the field by establishing benchmarks for evaluating various DRL algorithms in complex terrains.