EndoRD-GS : robust deformable endoscopic scene reconstruction via Gaussian splatting

Abstract

Real-time and realistic reconstruction of 3D dynamic surgical scenes from surgical videos is a novel and unique tool for surgical planning and intraoperative guidance. The 3D Gaussian splatting (GS), with its high rendering speed and reconstruction fidelity, has recently emerged as a promising technique for surgical scene reconstruction. However, existing GS-based methods still have two obvious shortcomings for realistic reconstruction. First, they largely struggle to capture localized yet intricate soft tissue deformations caused by complex instrument-tissue interactions. Second, they fail to model spatiotemporal coupling among Gaussian primitives for global adjustments during rapid perspective transformations, resulting in unstable reconstruction outputs. In this paper, we propose EndoRD-GS, an innovative approach that overcomes these two limitations through two core techniques: 1) periodic modulated Gaussian functions and 2) a new Biplane module. Specifically, our periodic modulated Gaussian functions incorporate meticulously designed modulations, significantly enhancing the representation of complex local tissue deformations. On the other hand, our Biplane module constructs spatiotemporal interactions among Gaussian primitives, enabling global adjustments and ensuring reliable scene reconstruction during rapid perspective transformations. Extensive experiments on three datasets demonstrate that our EndoRD-GS achieves superior performance in endoscopic scene reconstruction compared to state-of-the-art methods. The code is available at EndoRD-GS