Federated reinforcement learning for short-time scale operation of wind-solar-thermal power network with nonconvex models

Abstract

To schedule power sources operated by different entities in a short-time scale considering nonconvex generation cost and deep peak regulation (DPR) service constraints, this paper proposes an FRL-based multiple power sources coordination framework in wind-solar-thermal power network. In the studied power transmission network (TN), renewable energy sources and thermal power units connected to the same bus are aggregated as a wind-solar-thermal virtual power plant (WSTVPP). The transmission system operator (TSO) sends dispatch instructions to each WSTVPP by optimal power flow program, and allocates the cost of DPR service in TN. Based on the dispatch instruction, the internal power sources of each WSTVPP are scheduled by its local center control agent to achieve local economic operation while maximizing the overall DPR service revenue for the WSTVPP from the auxiliary service market. The multiple WSTVPPs operation is modeled as a partially observable Markov decision process, and solved by a designed FRL algorithm. The FRL algorithm employs a global neural network (NN) model for coordination, heterogeneous local NN models and data to efficiently train each WSTVPP control agent with individual objectives for handling multiple power sources scheduling in TN while preserving local privacy. Numerical studies validate the effectiveness of the proposed framework for handling the short-time scale power sources operation with nonconvex constraints.