An adaptive cutoff frequency selection approach for fast fourier transform method and its application into short-term traffic flow forecasting

Abstract

Historical measurements are usually used to build assimilation models in sequential data assimilation (S-DA) systems. However, they are always disturbed by local noises. Simultaneously, the accuracy of assimilation model construction and assimilation forecasting results will be affected. The fast Fourier transform (FFT) method can be used to acquire de-noised historical traffic flow measurements to reduce the influence of local noises on constructed assimilation models and improve the accuracy of assimilation results. In the practical signal de-noising applications, the FFT method is commonly used to de-noise the noisy signal with known noise frequency. However, knowing the noise frequency is difficult. Thus, a proper cutoff frequency should be chosen to separate high-frequency information caused by noises from the low-frequency part of useful signals under the unknown noise frequency. If the cutoff frequency is too high, too much noisy information will be treated as useful information. Conversely, if the cutoff frequency is too low, part of the useful information will be lost. To solve this problem, this paper proposes an adaptive cutoff frequency selection (A-CFS) method based on cross-validation. The proposed method can determine a proper cutoff frequency and ensure the quality of de-noised outputs for a given dataset using the FFT method without noise frequency information. Experimental results of real-world traffic flow data measurements in a sub-area of a highway near Birmingham, England, demonstrate the superior performance of the proposed A-CFS method in noisy information separation using the FFT method. The differences between true and predicted traffic flow values are evaluated using the mean absolute error (MAE), root mean square error (RMSE), and mean absolute percentage (MAPE) values. Compared to the results of the two commonly used de-noising methods, i.e., discrete wavelet transform (DWT) and ensemble empirical mode decomposition (EEMD) methods, the short-term traffic flow forecasting results of the proposed A-CFS method are much more reliable. In terms of the MAE value, the average relative improvements of the assimilation model built using the proposed method are 19.26%, 3.47%, and 4.25%, compared to the model built using raw data, DWT method, and EEMD method, respectively; the corresponding average relative improvements in RMSE are 19.05%, 5.36%, and 3.02%, respectively; lastly, the corresponding average relative improvements in MAPE are 18.88%, 2.83%, and 2.28%, respectively. The test results show that the proposed method is effective in separating noises from historical measurements and can improve the accuracy of assimilation model construction and assimilation forecasting results.