Multi-Person Image Matting by Using Pose-Estimation

다국어 초록 (Multilingual Abstract)

In this paper, we propose a transformer-based model for multi-person human image matting model using pose estimation, which uses information from human pose estimation to guide a transformer-based matting network to improve its performance in human image matting task. Our method takes a matting encoder and a multi-person human pose encoder to extract features from input image, then uses a fusion block to combine and merge these features, finally these merged features will be sent to ResNet decoder to be transformed into matting alpha map. Since most of existing image matting datasets are not focused on real world multi-person human matting, we build a combined human image dataset based on existing datasets and trained our model on this new dataset. Experiments shows that our method achieves high performance on multi-person human image matting task, especially on challenging cases.

목차 (Table of Contents)

• 1 Introduction 1
• 2 Related Work 5
• 2.1 Image Matting 5
• 2.2 Pose Estimation 6
• 3 Method 9
• 3.1 Overview 9
• 3.2 Multi-person Image Matting Network Architecture 10
• 3.2.1 Matting Encoder 11
• 3.2.2 Pose Encoder 12
• 3.2.3 Fusion Block 14
• 3.2.4 Decoder 16
• 3.3 Our Dataset 18
• 3.3.1 Introduction of Existing Datasets 18
• 3.3.2 Issues of Existing Datasets 19
• 3.3.3 Our New Dataset 20
• 4 Experiments and Results 22
• 4.1 Implementation Details 22
• 4.2 Results 23
• 5 Conclusion 27