An integrated deep reinforcement learning-linear control strategy for longitudinal control of connected and automated vehicles

Abstract

String stability is important to maintain the longitudinal control of connected and automated vehicles (CAVs). It prevents the amplification of the perturbations as they propagate through the platoon. A variety of methods based on the deep reinforcement learning (DRL) approach have been proposed for longitudinal control of CAVs, which show excellent performance. However, none of those methods consider string stability on theoretical grounds due to the lack of explicit mathematical models in the DRL approach. To address this problem, we integrate a novel linear controller in a DRL framework for longitudinal control of CAVs, referred to integrated DRL-linear control (IDL) strategy. It can guarantee string stability while striking a good balance among various benefits, including vehicle safety, comfort, and efficiency. We employ the twin delay depth deterministic policy gradient (TD3) algorithm, a promosing DRL, in the proposed framework for decision. Numerical simulation results demonstrate that the proposed approach ensures theoretical string stability while significantly enhancing vehicle safety, comfort, and efficiency compared to human-driven vehicles (HDVs) and a model-based cooperative adaptive cruise control (CACC) strategy. It also outperforms the deep deterministic policy gradient (DDPG) and pure TD3 strategies in terms of safety, comfort, and string stability. These results indicate that the proposed IDL strategy not only benefits from the advantages of the linear controller in analyzing theoretical string stability conditions but also retains the advantage of the DRL approach in terms of optimizing the trade-off between multiple benefits.