Construction of high-resolution lunar DEMs by fusing photogrammetry and shape-from-shading

Abstract

3D topographic modelling of the lunar surface is fundamental to lunar and planetary science and exploration. Topographic models such as digital elevation models (DEMs) of the lunar surface have seen extensive use in the study of the lunar surface and geologic processes and for planning of lunar exploration missions. Lunar DEMs are usually generated from remote sensing data acquired by instruments such as cameras and laser altimeters onboard lunar orbiters. The currently available lunar DEMs have the following limitations: (1) lunar DEMs with global or near-global coverage usually have a comparatively low spatial resolution (i.e., 60 m/pixel or lower); (2) regional high-resolution DEMs generated with photogrammetry have limited spatial coverage; and (3) lunar DEMs cannot reach the effective spatial resolution of optical images acquired by lunar orbiters. The performance of photogrammetry, in particular, depends heavily on the robustness of image matching, which directly affects the spatial resolution and accuracy of the resulting lunar DEMs. Photogrammetry may even fail due to the limitations of the image matching approach when the stereo images are poorly textured and/or acquired with large variations in illumination, which are common problems for lunar images. To overcome these limitations, lunar topographic modelling was performed with Shape-from-Shading (SfS; also known as photoclinometry), a technique that generates 3D models based on the image's photometric content (i.e., intensity value). It can reconstruct lunar DEMs with optimal spatial resolution using single or multiple images, and it is robust to fine-grained details, subtle surface textures and variations in illumination. However, SfS lacks accuracy for large-scale topographic modelling, at which photogrammetry excels. As a result, SfS and photogrammetry are natural complements for the generation of accurate and optimal 3D representations of the lunar surface, which can better support exploration missions (e.g., landing site selection) and scientific research (e.g., lunar geology). This research aims to integrate SfS and photogrammetry to construct accurate lunar DEMs with optimal spatial resolution. This study includes three major developments. First, a novel hierarchical SfS approach was developed. Given a single monocular image, the single-image SfS (SI-SfS) approach refines an initial DEM of lower resolution to an optimal resolution of the image. The corresponding pixel-wise albedo map of the surface is estimated simultaneously in the process and is used by SI-SfS to regularise the 3D refinement of the initial DEM. A hierarchical architecture was used for effective SfS at multiple resolutions, and an explicit shadow constraint was developed and incorporated into the SI-SfS approach to overcome image regions with shadows. Experiments were carried out using Narrow Angle Camera (NAC) images from the Lunar Reconnaissance Orbiter Camera (LROC), with a spatial resolution of 0.5 to 1.5 m/pixel. The integrated Selenological and Engineering Explorer and LRO Elevation Model (SLDEM), with a spatial resolution of 60 m/pixel, was chosen as the initial DEM for constraint. The results indicate that local topographic details were well recovered by the SI-SfS approach with plausible albedo estimation. The refined DEM achieved geometric accuracies of 2 to 7 m relative to the reference NAC DEMs generated from photogrammetry. The low-frequency topographic consistency depends upon the quality of the low-resolution DEM and the difference in the spatial resolution between the image and the initial DEM. Second, a photometric stereo SfS (PS-SfS) approach was developed. The PS-SfS approach uses two co-registered images acquired under different illumination conditions to generate high-resolution lunar DEMs. In contrast to the previous SI-SfS approach, PS-SfS does not require an initial DEM. A novel formulation was used to effectively factor out the effects of albedo variations, leading to more robust performance. Moreover, a quantitative investigation of the effects of illumination differences on the performance of 3D reconstruction was carried out for the PS-SfS. First, the fundamental process of PS-SfS was mathematically modelled to correlate the numeric solution of PS-SfS and the images' differences in illumination. Based upon the mathematical model, an error model was then derived to analyse the relationships between the azimuthal and zenith angles of the images' illumination and the reconstruction qualities. The developed PS-SfS approach and the error model were verified with LROC NAC images. Our experimental analyses reveal that the resulting error in PS-SfS depends upon both the azimuthal and the zenith angles of illumination and upon the general intensity of the images. The predictions from the proposed error model are consistent with the actual slope errors obtained by PS-SfS using LROC NAC images. The proposed error model enriches the theory of PS-SfS and is significant for optimised lunar surface reconstruction based on SfS techniques. Third, an integrated photogrammetric and photoclinometric approach was developed. The integrated approach can generate lunar DEMs with optimal resolution and is invariant to variations in illumination. The fusion of photoclinometry and photogrammetry involves two main steps. First, a photoclinometry-assisted image matching (PAM) approach is developed by integrating PS-SfS into the image matching stage to create pixel-wise matches, even for images with large differences in illumination. Second, the DEM derived from photogrammetry using the matching results is refined to optimal resolution using the SI-SfS approach. The proposed approach has been validated with high-resolution LROC NAC images acquired in various conditions of illumination at the Chang'E-4 and Chang'E-5 landing sites. The results indicate that the integrated approach is robust to severe inconsistencies in illumination and to subtle textures in cases in which the conventional approaches fail. The integrated approach can achieve geometric accuracies comparable to photogrammetry, while giving more small-scale topographic detail. The presented research and development allows for the generation of high-precision and high-resolution lunar DEMs from a single image or multiple images, which effectively extends the ability of photogrammetry in lunar mapping. The developed approaches have been widely used in China's Chang'E-4 and Chang'E-5 lunar missions to generate high-resolution DEMs of the landing regions to support optimised landing site selection and surface operation. The developed approaches can also be used for high-resolution topographic mapping of other planetary bodies, such as Mercury and asteroids, and to provide a useful reference for topographic mapping on Earth.