A generative whole-brain segmentation model for positron emission tomography images

Abstract

Purpose: Whole-brain segmentation via positron emission tomography (PET) imaging is crucial for advancing neuroscience research and clinical medicine, providing essential insights into biological metabolism and activity within different brain regions. However, the low resolution of PET images may have limited the segmentation accuracy of multiple brain structures. Therefore, we propose a generative multi-object segmentation model for brain PET images to achieve automatic and accurate segmentation. Methods: In this study, we propose a generative multi-object segmentation model for brain PET images with two learning protocols. First, we pretrained a latent mapping model to learn the mapping relationship between PET and MR images so that we could extract anatomical information of the brain. A 3D multi-object segmentation model was subsequently proposed to apply whole-brain segmentation to MR images generated from integrated latent mapping models. Moreover, a custom cross-attention module based on a cross-attention mechanism was constructed to effectively fuse the functional information and structural information. The proposed method was compared with various deep learning-based approaches in terms of the Dice similarity coefficient, Jaccard index, precision, and recall serving as evaluation metrics. Results: Experiments were conducted on real brain PET/MR images from 120 patients. Both visual and quantitative results indicate that our method outperforms the other comparison approaches, achieving 75.53% ± 4.26% Dice, 66.02% ± 4.55% Jaccard, 74.64% ± 4.15% recall and 81.40% ± 2.30% precision. Furthermore, the evaluation of the SUV distribution and correlation assessment in the regions of interest demonstrated consistency with the ground truth. Additionally, clinical tolerance rates, which are determined by the tumor background ratio, have confirmed the ability of the method to distinguish highly metabolic regions accurately from normal regions, reinforcing its clinical applicability. Conclusion: For automatic and accurate whole-brain segmentation, we propose a novel 3D generative multi-object segmentation model for brain PET images, which achieves superior model performance compared with other deep learning methods. In the future, we will apply our whole-brain segmentation method to clinical practice and extend it to other multimodal tasks.