Communications in Information and Systems

Volume 23 (2023)

Number 1

Multi-foreground objects segmentation based on RGB-D image

Pages: 31 – 55

DOI: https://dx.doi.org/10.4310/CIS.2023.v23.n1.a2

Authors

Yan Li (School of Information Science and Engineering, Shandong University, QingDao, China)

Di Zhu (School of Information Science and Engineering, Shandong University, QingDao, China)

Hui Chen (School of Information Science and Engineering, Shandong University, QingDao, China)

Jing Nie (Shandong Institute of Science and Technology Information, Jinan, China)

JiaJu Liu (Shandong International Trust Co., Ltd., Jinan, China)

ChangHe Tu (School of Computer Science and Technology, Shandong University, QingDao, China)

HaiKun Li (School of Information Science and Engineering, Shandong University, QingDao, China)

Abstract

Silhouette extraction of foreground objects appears frequently in various real-world applications, such as Advanced Driving Assistant System, Intelligent Monitoring System, and movie production. Plenty of solutions have been developed to extract silhouette in RGB image with only color information. Since those color based silhouette extraction methods still have difficulties to separate overlapping foreground objects and eliminate excessive segmentation, this paper proposes a novel object segmentation method using color and depth information in RGB-D images. Firstly, we remove the ground plane using the normal map of depth image. Secondly, to separate foreground objects at different distances completely and correctly, the deep Residual Network (ResNets) and Otsu’s multithresholding method are combined to divide the depth image into multiple layers. Each depth layer contains only one foreground object or objects at same distance. Finally, the outline of foreground object is extracted directly from its depth layer, and refined with color information. Experimental results demonstrate that our method has a better performance than those using color or depth information only, and extracts more types of objects than neural networks.

Keywords

depth layer, RGB-D, multi-object segmentation, ResNet

Full Text (PDF format)

This work was supported by the Key Project of Science and Technology Development Plan of Shandong Province of China under Grant No. 2019GGX101018.

Received 28 February 2022

Published 17 April 2023