Subgroup analysis in the heterogeneous Cox model

Abstract

In the analysis of censored survival data, to avoid a biased inference of treatment effects on the hazard function of the survival time, it is important to consider the treatment heterogeneity. Without requiring any prior knowledge about the subgroup structure, we propose a data driven subgroup analysis procedure for the heterogeneous Cox model by constructing a pairwise fusion penalized partial likelihood-based objective function. The proposed method can determine the number of subgroups, identify the group structure, and estimate the treatment effect simultaneously and automatically. A majorized alternating direction method of multipliers algorithm is then developed to deal with the numerically challenging high-dimensional problems. We also establish the oracle properties and the model selection consistency for the proposed penalized estimator. Our proposed method is evaluated by simulation studies and further illustrated by the analysis of the breast cancer data.