Please use this identifier to cite or link to this item:
Title: SV-RCNet : workflow recognition from surgical videos using recurrent convolutional network
Authors: Jin, YM
Dou, Q
Chen, H
Yu, LQ
Qin, J 
Fu, CW
Heng, PA
Keywords: Recurrent convolutional network
Surgical workflow recognition
Joint learning of spatio-temporal features
Very deep residual network
Long short-term memory
Issue Date: 2018
Publisher: Institute of Electrical and Electronics Engineers
Source: IEEE transactions on medical imaging, May 2018, v. 37, no. 5, p. 1114-1126 How to cite?
Journal: IEEE transactions on medical imaging 
Abstract: We propose an analysis of surgical videos that is based on a novel recurrent convolutional network (SV-RCNet), specifically for automatic workflow recognition from surgical videos online, which is a key component for developing the context-aware computer-assisted intervention systems. Different from previous methods which harness visual and temporal information separately, the proposed SV-RCNet seamlessly integrates a convolutional neural network (CNN) and a recurrent neural network (RNN) to forma novel recurrent convolutional architecture in order to take full advantages of the complementary information of visual and temporal features learned from surgical videos. We effectively train the SV-RCNet in an end-to-end manner so that the visual representations and sequential dynamics can be jointly optimized in the learning process. In order to produce more discriminative spatio-temporal features, we exploit a deep residual network (ResNet) and a long short term memory (LSTM) network, to extract visual features and temporal dependencies, respectively, and integrate them into the SV-RCNet. Moreover, based on the phase transition-sensitive predictions from the SV-RCNet, we propose a simple yet effective inference scheme, namely the prior knowledge inference (PKI), by leveraging the natural characteristic of surgical video. Such a strategy further improves the consistency of results and largely boosts the recognition performance. Extensive experiments have been conducted with the MICCAI 2016 Modeling and Monitoring of Computer Assisted Interventions Workflow Challenge dataset and Cholec80 dataset to validate SV-RCNet. Our approach not only achieves superior performance on these two datasets but also outperforms the state-of-the-art methods by a significant margin.
ISSN: 0278-0062
EISSN: 1558-254X
DOI: 10.1109/TMI.2017.2787657
Appears in Collections:Journal/Magazine Article

View full-text via PolyU eLinks SFX Query
Show full item record


Citations as of Apr 3, 2019


Last Week
Last month
Citations as of Apr 6, 2019

Page view(s)

Citations as of May 21, 2019

Google ScholarTM



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.