Propensity score methods for road safety evaluation : practical suggestions from a simulation study

Abstract

The propensity score (PS) based method has been increasingly used in road safety evaluation studies. However, several major considerations regarding its implementation arise when using the PS method. First, as is well known, the PS method is ‘data hungry’ in terms of the number of treated and control units, however, it is sometimes difficult and time-consuming to construct a large sample in road safety studies. It would be helpful to better understand how to choose a proper sample size, as well as the ratio of the number of treated units to the control ones. Second, the criteria used for covariates selection of the PS model were not fully consistent across the existing road safety evaluation studies. Due to the complicated mechanisms behind the implementation of road safety measures and policies, including all relevant covariates that affect both the selection into treatment (i.e., implementation of road safety measures) and the outcomes (i.e., road accidents) is impossible. In this paper, we conduct a simulation study to investigate such issues and provide some practical suggestions for using PS methods in road safety evaluations. The estimator considered in this study is the inverse probability weighting estimator based on the PS. Our results suggest that the bias and variance of the estimated treatment effect will remain stable when the sample size reaches a certain level. A proper sample size is the one that ensures relevant covariates achieve acceptable balance. Regarding the issue of covariates selection, including the covariates that significantly affect the road accidents is recommended, regardless of whether they affect the implementation of road safety measures. This study also proposes practical procedures for using the weighting approach to evaluate the effects of road safety treatments.