Graph based image segmentation and evaluation

Abstract

Image segmentation is a fundamental yet still challenging problem in computer vision and image processing. In this thesis, we address this problem in light of the graph-based theory. An interactive and an automatic segmentation algorithms are proposed to partition the image into meaningful disjoint regions. For the interactive segmentation algorithm, we present an iterated region merging based graph cuts algorithm which is a novel extension of the standard graph cuts algorithm. The proposed algorithm starts from the user labeled sub-graph and works iteratively to label the surrounding un-segmented regions. The sub-graph requires less user guidance for segmentation and thus better results can be obtained under the same amount of user interaction. The proposed automatic segmentation algorithm is working in a region merging style. With an initially over-segmented image, it is performed by iteratively merging the regions according to a statistical test. There are two essential issues in a region merging algorithm: order of merging and the stopping criterion. These two issues are solved by a novel predicate, which is defined by the sequential probability ratio test (SPRT) and the minimal cost criterion. In addition, a faster algorithm is developed to accelerate the region merging process, which maintains a nearest neighbor graph (NNG) in each iteration. Moreover, we consider the problem of evaluating the quality of image segmentations, which is an indispensable step for choosing an appropriate output of the image segmentation algorithms. We assume that if a segmentation is "good", it can be constructed by some pieces of the ground truth segmentations. For a given segmentation, we construct adaptively a new ground truth which can be locally matched to the segmentation as much as possible and preserve the structural consistency in the ground truths. We use the structure-based and the boundary-based measures to validate its effectiveness. To the best of our knowledge, this is the first work that provides a framework to adaptively combine multiple ground truths for quantitative segmentation evaluation. Experiments are conducted on the benchmark images and the results show that the proposed method can faithfully reflect the perceptual qualities of segmentations.