Road-adaptive precise path tracking based on reinforcement learning method

Abstract

This paper proposes a speed-adaptive autonomous driving path-tracking framework based on the soft actor–critic (SAC) and pure pursuit (PP) methods, named the SACPP controller. The framework first analyzes the obstacles around the vehicle and plans an obstacle-free reference path with the minimum curvature using the hybrid A* algorithm. Next, based on the generated reference path, the current state of the vehicle, and the vehicle motor energy efficiency diagram, the optimal speed is calculated in real time, and the vehicle dynamics preview point at the future moment—specifically, the look-ahead distance—is predicted. This process relies on the learning of the SAC network structure. Finally, PP is used to generate the front wheel angle control value by combining the current speed and the predicted preview point. In the second layer, we carefully designed the evaluation function in the tracking process based on the uncertainties and performance requirements that may occur during vehicle driving. This design ensures that the autonomous vehicle can not only quickly and accurately track the path, but also effectively avoid surrounding obstacles, while keeping the motor running in the high-efficiency range, thereby reducing energy loss. In addition, since the entire framework uses a lightweight network structure and a geometry-based method to generate the front wheel angle, the computational load is significantly reduced, and computing resources are saved. The actual running results on the i7 CPU show that the control cycle of the control framework exceeds 100 Hz.