Upscaling infrastructure asset management systems under complex uncertainties to network assets using machine learning

Abstract

Deteriorating and at-risk infrastructure assets should be maintained at acceptable conditions by infrastructure asset management (IAM) systems to ensure the safety and welfare of communities. Given the importance of maintaining infrastructure assets, asset management at both project- and network levels has been the focus of hundreds of studies. These IAM systems utilize probabilistic and non-linear models to accurately model various phenomena. With a commonly adopted framework using Monte Carlo simulation and heuristic optimization algorithms, IAM systems proposed in the literature aim to maintain the functionality of assets in their life cycle by optimally allocating limited resources to different intervention actions in time. Although these studies and previously developed asset management systems have pushed the boundaries of this field, they are subject to a number of limitations. First, these advances have been made separately without the ability to build upon one another. Second, project- level IAM (PL-IAM) is capable of optimizing maintenance interventions in the life cycle of assets by incorporating probabilistic and complex models but at the expense of relatively high computation time. Due to their high computational costs, upscaling complex PL-IAM systems to a multitude of assets is currently far from practical. Consequently, uncertainties have usually been simplified in this problem to reach acceptable strategies in feasible computational time. This simplification could lead to sub-optimal and far from reality strategies. Also, the derived optimal plans lack flexibility in decision-making in face of unprecedented uncertainties. Ever-changing uncertain phenomena might render previously stochastic models obsolete and, therefore, optimized MRR plans sub-optimal. In a more recent line of research, reinforcement learning (RL) has been adopted by IAM researchers for adding flexibility regarding uncertainties in preventive actions decision-making. As the third major limitation highlighted in this dissertation, this line of research lacks (1) incorporating other sources of uncertainties, such as hazards, apart from deterioration patterns, and (2) considering managerial aspects of IAM such as stakeholders' utilities. Fourth, the network-level infrastructure asset management (NL-IAM) problem, which is excessively large and complex due to a large number of decision parameters, possible strategies, and underlying uncertainties, has not gained proper attention in the RL-based IAM studies. This dissertation puts forward the first open-source, extensible, freely accessible, and modular platform developed in Python, GIAMS, paving the way for researchers and practitioners to easily collaborate and contribute to future related research. Drawing upon related literature, it describes modules of GIAMS and illustrates their use with bridge components and models. Aiming to address the gap between the literature and practice of PL­IAM, which is making complex PL-IAM systems computationally applicable to all assets in a network, this dissertation presents a methodology to replace the time-consuming simulation modules of optimization algorithms with a trained machine learning model estimating life cycle costs analysis (LCCA) results. Then, a new machine learning-based methodology is presented to learn the behavior of optimization algorithms and estimate (near-)optimal intervention timings instead of doing the optimization. Next, this study provides a holistic framework, from the early development of IAM systems and microworlds to training RL models and verification of results. This framework focuses on considering deterioration, hazards, and costs fluctuations as the main sources of uncertainties and adopting managerial aspects into decision making. Finally, one of the first RL-based NL-IAM systems is put forward for achieving intervention strategies while considering different sources of uncertainties such as earthquake, deterioration, and costs fluctuations. One project-level lifecycle optimization and one network-level project selection based on the Indiana, US, bridge network with more than 4,600 bridges are provided to demonstrate the applicability of GIAMS. Then, deep neural network (DNN) models were trained on the LCCA results of more than 1.4 million semi-synthesized bridges based on the US NBI considering different intervention actions and uncertainties about condition ratings, hazards, and costs. The findings show that the trained DNN models can accurately estimate the complex LCCA results 5 orders of magnitudes faster than simulation techniques. As the next step, an ensemble of random forests models was trained on optimal maintenance timings of more than 1.6 million semi-synthesized bridges to illustrate the proposed methodology for directly estimating optimal plans. The trained model could yield optimized MRR plans, with more than 95% accuracy on the test set and more than 89% accuracy on the real highway bridges of Indiana with more than 4600 assets, 6 orders of magnitudes faster than the conventional framework of complex MRR optimization. Given the proposed RL-based framework for IAM, multi-agents RL models, based on DQN and actor-critic models, are constructed and trained for taking intervention actions regarding elements of a real bridge in Indiana, the US, through its life cycle. Both models could increase the utilities by up to 14% and decrease the costs in the project-level analysis. Alongside the flexibility in decision-making provided by the trained A2C models for NL-IAM, the results show that more favorable strategies in terms of stakeholders' utilities could be achieved using the proposed framework. Effective solutions for tackling the curse of dimensionality and reward engineering are also provided herein. The modular and open-source nature of the presented software (GIAMS) makes it readily capable of further extension in a variety of asset management topics. The proposed methodology for estimating LCCA results help practitioners reduce the optimization and LCCA computation times of complex IAM systems to a feasible level for practical utilization. Practitioners can adopt the proposed methodology for predicting optimal plans to enhance their decision-making systems, obtain optimal maintenance plans without sacrificing complex and accurate models, and take another step towards sustainability objectives. Consistent with the existing practice of IAM, the proposed RL-based framework brings flexibility in face of uncertainties to the IAM decision-making process. In other words, the trained MARL models can assist decision-makers with making decisions that are close to reality while considering complex models in a short time in the IAM domains. The proposed framework and solutions for RL-based NL-IAM can pave the path for researchers toward enhancing the current state of NL-IAM frameworks. Trained multi-agent A2C models can inform asset managers with potentially more profitable, more desirable, and closer to reality strategies leading to community-level enhancements in sustainability measures.