Pixelwise Gradient Model for Image Fusion (PGMIF) : a multi-sequence magnetic resonance imaging (MRI) fusion model for tumor contrast enhancement of nasopharyngeal carcinoma

Abstract

Background: Different image modalities capture different aspects of a patient. It is desirable to produce images that capture all such features in a single image. This research investigates the potential of multi-modal image fusion method to enhance magnetic resonance imaging (MRI) tumor contrast and its consistency across different patients, which can capture both the anatomical structures and tumor contrast clearly in one image, making MRI-based target delineation more accurate and efficient. Methods: T1-weighted (T1-w) and T2-weighted (T2-w) magnetic resonance (MR) images from 80 nasopharyngeal carcinoma (NPC) patients were used. A novel image fusion method, Pixelwise Gradient Model for Image Fusion (PGMIF), which is based on the pixelwise gradient to capture the shape and a generative adversarial network (GAN) term to capture the image contrast, was introduced. PGMIF is compared with several popular fusion methods. The performance of fusion methods was quantified using two metrics: the tumor contrast-to-noise ratio (CNR), which aims to measure the contrast of the edges, and a Generalized Sobel Operator Analysis, which aims to measure the sharpness of edge. Results: The PGMIF method yielded the highest CNR [median (mdn) =1.208, interquartile range (IQR) =1.175–1.381]. It was a statistically significant enhancement compared to both T1-w (mdn =1.044, IQR =0.957–1.042, P<5.60×10−4) and T2-w MR images (mdn =1.111, IQR =1.023–1.182, P<2.40×10−3), and outperformed other fusion models: Gradient Model with Maximum Comparison among Images (GMMCI) (mdn =0.967, IQR =0.795–0.982, P<5.60×10−4), Deep Learning Model with Weighted Loss (DLMWL) (mdn =0.883, IQR =0.832–0.943, P<5.60×10−4), Pixelwise Weighted Average (PWA) (mdn =0.875, IQR =0.806–0.972, P<5.60×10−4) and Maximum of Images (MoI) (mdn =0.863, IQR =0.823–0.991, P<5.60×10−4). In terms of the Generalized Sobel Operator Analysis, a measure based on Sobel operator to measure contrast enhancement, PGMIF again exhibited the highest Generalized Sobel Operator (mdn =0.594, IQR =0.579–0.607; mdn =0.692, IQR =0.651–0.718 for comparison with T1-w and T2-w images), compared to: GMMCI (mdn =0.491, IQR =0.458–0.507, P<5.60×10−4; mdn =0.495, IQR =0.487–0.533, P<5.60×10−4), DLMWL (mdn =0.292, IQR =0.248–0.317, P<5.60×10−4; mdn =0.191, IQR =0.179–0.243, P<5.60×10−4), PWA (mdn =0.423, IQR =0.383–0.455, P<5.60×10−4; mdn =0.448, IQR =0.414–0.463, P<5.60×10−4) and MoI (mdn =0.437, IQR =0.406–0.479, P<5.60×10−4; mdn =0.540, IQR =0.521–0.636, P<5.60×10−4), demonstrating superior contrast enhancement and sharpness compared to other methods. Conclusions: Based on the tumor CNR and Generalized Sobel Operator Analysis, the proposed PGMIF method demonstrated its capability of enhancing MRI tumor contrast while keeping the anatomical structures of the input images. It holds promises for NPC tumor delineation in radiotherapy.