A reinforcement learning approach for control of window behavior to reduce indoor PM₂.₅ concentrations in naturally ventilated buildings

Abstract

Smart control of window behavior is a means of effectively reducing concentrations of indoor PM₂.₅ (particulate matter with aerodynamic diameter less than 2.5 μm) in naturally ventilated residential buildings without indoor air cleaning devices. This study aimed to develop a reinforcement learning approach to automatically control window behavior in real time for mitigation of indoor PM₂.₅ pollution. The proposed method trains the window controller with the use of a deep Q-network (DQN) in a specific naturally ventilated apartment in the course of a month. The trained controller can then be employed to control window behavior in order to reduce the indoor PM₂.₅ concentrations in that apartment. The required input data for the controller are the real-time indoor and outdoor PM₂.₅ concentrations with a 1-min resolution, which can easily be obtained with low-cost sensors available on the market. A series of simulations were conducted in a virtual typical apartment in Beijing and a real apartment in Tianjin. The results show that, compared with the baseline I/O ratio algorithm, the proposed reinforcement learning window-control algorithm reduced the average indoor PM₂.₅ concentration by 12.80% in a one-year period. Furthermore, the proposed algorithm reduced the indoor PM₂.₅ concentrations in the real apartment by 9.11% when compared with the I/O ratio algorithm and by 7.40% when compared with real window behavior.