Prompt detection and classification for excavated soils using multi-modal data deep learning

Abstract

According to the 2018 survey from China's Ministry of Housing and Urban-Rural Development, the annual production of excavated soils exceeded 1.91 billion tons, accounting for 31.4% of the total solid waste output. Sustainable management and large-scale utilization of the soil have thus become critical challenges in safe solid waste disposal. Developed countries have explored disposal strategies for excavated soil, such as reusing sandy soils for backfilling, repurposing silty soils for mine pit rehabilitation, and producing construction materials from clayey soils. Given the significant variability in engineering and mineralogical properties of soils, developing prompt classification methods based on attribute characterization is essential for improving management of excavated soils with high production. This study aims to achieve soil prompt detection and efficient classification based on engineering and resource attributes before the disposal stage. First, the research investigated correlations between soil images (visual indicators), cone index (mechanical indicators), time-domain reflectometry (TDR) parameters (electrical indicators), and basic soil properties using digital image processing and multivariate OLS regression. Gray scale distribution negatively correlates with sand content but positively with clay content. Warm hues (Y/R) correlate with coarse-grained content, while cool hues (G) correlate with fine-grained content. Dielectric constant and electrical conductivity are higher in fine-grained soils, positively correlates with moisture but negatively with organic content. Electrical conductivity positively correlates with montmorillonite content but negatively correlates with kaolinite content. The potentials of soil images, cone index, and TDR parameters were identified. Subsequently a cone penetrometer based on TDR and laser-induced fluorescence (LIF) was developed. The LIF module with 325 nm ultraviolet light and 275 nm ultraviolet light was to detect underground dissolved organic matters and polycyclic aromatic hydrocarbons respectively, while the TDR module was to detect subsoil electrical parameters. An Excavated Soil Information Collecting System (ESICS) based on TDR cone penetrometer and digital camera was established at Xiecun Wharf, the largest soil transferring platform in China, synchronously capturing soil multiple indices within 50 seconds. 3,243 groups of multi-source heterogenous data were obtained, including soil images, spatial series data of cone index (CI) curves, time series data of TDR waveforms. After data augmentation, a big dataset containing 23,122 sets with labels based on the Unified Soil Classification System, moisture content, and mineral composition were created. Then multi-modal deep learning fusion models were developed for efficient classification, employing 7 early fusions, 3 intermediate fusions, and 2 late fusion strategies. Performance metrics were evaluated, including loss, accuracy, precision, recall, specificity, and AUROC. The best-performing model used two-stage intermediate fusion with five modalities, achieving over 99% accuracy. Generative diffusion models, specifically a Denoising Diffusion Probabilistic Model, were used to explore cross-modal relationships. Performance metrics such as SSIM, LPIPS, and RMSE were employed to analyze the model's training results. Key findings reveal that soil images encode water content information, which correlates with CI curves and TDR waveforms, CI and TDR data cannot capture color-based mineral composition details from images, and TDR waveforms uniquely detect pollution indicators (e.g., electrical conductivity), undetectable via other methods. Finally, above technologies were applied in Wenzhou city, China. By using a novel approach for quantifying excavated soil and construction slurry production based on multi-source information, such as the urban topographic map, geological survey reports, urban master plan, and remote sensing images, the 107.5 million m³ of soil and 81.7 million m³ of slurry in Wenzhou city from 2021 to 2025 were estimated.