Please use this identifier to cite or link to this item:
Title: Stochastic optimal CPS relaxed control methodology for interconnected power systems using Q-learning method
Authors: Yu, T
Zhou, B
Chan, KW 
Lu, E
Keywords: Automatic generation control
China Southern Power Grid
Control performance standard
Markov decision process
Optimal control
Q-learning algorithm
Reinforcement learning
Issue Date: 2011
Publisher: Asce-Amer Soc Civil Engineers
Source: Journal of energy engineering, 2011, v. 137, no. 3, p. 116-129 How to cite?
Journal: Journal of Energy Engineering 
Abstract: This paper presents the application and design of a novel stochastic optimal control methodology based on the Q-learning method for solving the automatic generation control (AGC) under the new control performance standards (CPS) for the North American Electric Reliability Council (NERC). The aims of CPS are to relax the control constraint requirements of AGC plant regulation and enhance the frequency dispatch support effect from interconnected control areas. The NERC's CPS-based AGC problem is a dynamic stochastic decision problem that can be modeled as a reinforcement learning (RL) problem based on the Markov decision process theory. In this paper, the Q-learning method is adopted as the RL core algorithm with CPS values regarded as the rewards from the interconnected power systems; the CPS control and relaxed control objectives are formulated as immediate reward functions by means of a linear weighted aggregative approach. By regulating a closed-loop CPS control rule to maximize the long-term discounted reward in the procedure of online learning, the optimal CPS control strategy can be gradually obtained. This paper also introduces a practical semisupervisory group prelearning method to improve the stability and convergence ability of Q-learning controllers during the prelearning process. Tests on the China Southern Power Grid demonstrate that the proposed control strategy can effectively enhance the robustness and relaxation property of AGC systems while CPS compliances are ensured.
DOI: 10.1061/(ASCE)EY.1943-7897.0000017
Appears in Collections:Journal/Magazine Article

View full-text via PolyU eLinks SFX Query
Show full item record


Last Week
Last month
Citations as of Nov 5, 2018


Last Week
Last month
Citations as of Sep 23, 2018

Page view(s)

Last Week
Last month
Citations as of Nov 11, 2018

Google ScholarTM



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.