Contour Model and Robust Segmentation based Human Pose Estimation in Images and Videos

보안공학연구지원센터(IJSIP) 보안공학연구지원센터 2015 International Journal of Signal Processing, Image Vol.8 No.3

Pose estimation which is regard as the cross-technology of computer vision and pattern recognition, and an important prerequisite for human behavior understanding. Human pose estimation which use the probability theory, machine learning, pattern recognition, graph theory and other theories to get the position, the deflection angle of the various parts of the body. Then make the detection and estimation parameters for the human body pose. When the image has interference in the background, color and scale changed, human pose complex, self-occlusion and interpersonal interaction occlusion may make the precision and accuracy of pose estimation face great challenge. Thus, according to the above problems, this paper use the advanced model of the human body as contour model to descript the complex pose, in order to make the model more accurate and suitable for various human pose, we pre-clustering the human body pose of the training samples before we trained the model and in order to ensure the accuracy of the pose we use robust segmentation of multi-view with a novel shape prior. The experiment shows that the algorithm performs better than the classic algorithm on the public datasets.

Mask R-CNN과 겹침 처리 알고리즘을 활용한 상단의 원형 객체 검출 및 자세 추정

윤석현,양정직,김청준,황면중 제어·로봇·시스템학회 2023 제어·로봇·시스템학회 논문지 Vol.29 No.12

This paper proposes an algorithm for detecting and estimating the pose of top objects in a complex environment where thin metal circular plates are randomly stacked. In complex environments where multiple instances of the same object are randomly stacked, the robot needs to detect and compare objects to identify the top ones for grasping. Our approach involves a combination of deep learning-based instance segmentation and an overlap handling algorithm for precise top object detection. Subsequently, leveraging three-dimensional geometric data, we estimate the object's pose by determining its plane. To validate the proposed algorithm, we constructed two environments consisting of objects with different sizes and thicknesses. The first experiment quantitatively validated the object detection and overlap handling algorithm. The second experiment quantitatively compared different plane estimation algorithms. The third experiment quantitatively compared the pose of objects using the G-ICP (Generalized Iterative Closest Point) algorithm and the proposed algorithm against the ground truth pose. Additionally, we performed a qualitative comparison by visualizing the poses estimated by each algorithm in the images. In the experimental results, the overlap handling algorithm had an average success rate of 84.21%. Additionally, pose estimation using G-ICP before plane estimation frequently resulted in issues like drift in the center point and frequent misalignment with areas other than the object. On the other hand, pose estimation using G-ICP after plane estimation and the proposed algorithm yielded similar performance with average ADD-S values of 6mm or less. However, the pose estimated using the proposed algorithm resulted in a minimum 0.25x reduction in execution time compared to the G-ICP algorithm.

2.5D human pose estimation for shadow puppet animation

( Shiguang Liu ),( Guoguang Hua ),( Yang Li ) 한국인터넷정보학회 2019 KSII Transactions on Internet and Information Syst Vol.13 No.4

Digital shadow puppet has traditionally relied on expensive motion capture equipments and complex design. In this paper, a low-cost driven technique is presented, that captures human pose estimation data with simple camera from real scenarios, and use them to drive virtual Chinese shadow play in a 2.5D scene. We propose a special method for extracting human pose data for driving virtual Chinese shadow play, which is called 2.5D human pose estimation. Firstly, we use the 3D human pose estimation method to obtain the initial data. In the process of the following transformation, we treat the depth feature as an implicit feature, and map body joints to the range of constraints. We call the obtain pose data as 2.5D pose data. However, the 2.5D pose data can not better control the shadow puppet directly, due to the difference in motion pattern and composition structure between real pose and shadow puppet. To this end, the 2.5D pose data transformation is carried out in the implicit pose mapping space based on self-network and the final 2.5D pose expression data is produced for animating shadow puppets. Experimental results have demonstrated the effectiveness of our new method.

2D–3D pose consistency-based conditional random fields for 3D human pose estimation

Chang, Ju Yong,Lee, Kyoung Mu Elsevier 2018 Computer vision and image understanding Vol.169 No.-

<P><B>Abstract</B></P> <P>This study considers the 3D human pose estimation problem in a single RGB image by proposing a conditional random field (CRF) model over 2D poses, in which the 3D pose is obtained as a byproduct of the inference process. The unary term of the proposed CRF model is defined based on a powerful heat-map regression network, which has been proposed for 2D human pose estimation. This study also presents a regression network for lifting the 2D pose to 3D pose and proposes the prior term based on the consistency between the estimated 3D pose and the 2D pose. To obtain the approximate solution of the proposed CRF model, the N-best strategy is adopted. The proposed inference algorithm can be viewed as sequential processes of bottom-up generation of 2D and 3D pose proposals from the input 2D image based on deep networks and top-down verification of such proposals by checking their consistencies. To evaluate the proposed method, we use two large-scale datasets: Human3.6M and HumanEva. Experimental results show that the proposed method achieves the state-of-the-art 3D human pose estimation performance.</P> <P><B>Highlights</B></P> <P> <UL> <LI> We propose a new CRF model with a novel 2D–3D pose consistency prior for 3D human pose estimation. </LI> <LI> We propose a simple but powerful 2D-to-3D pose lifting method based on MLP. </LI> <LI> We show the solution of the proposed CRF model can be efficiently found by the N-best strategy. </LI> <LI> Thorough experiments show that the proposed method achieves state-of-the-art 3D human pose estimation performance. </LI> </UL> </P>

Robust 2D human upper-body pose estimation with fully convolutional network

Lee, Seunghee,Koo, Jungmo,Kim, Jinki,Myung, Hyun Techno-Press 2018 Advances in robotics research Vol.2 No.2

With the increasing demand for the development of human pose estimation, such as human-computer interaction and human activity recognition, there have been numerous approaches to detect the 2D poses of people in images more efficiently. Despite many years of human pose estimation research, the estimation of human poses with images remains difficult to produce satisfactory results. In this study, we propose a robust 2D human body pose estimation method using an RGB camera sensor. Our pose estimation method is efficient and cost-effective since the use of RGB camera sensor is economically beneficial compared to more commonly used high-priced sensors. For the estimation of upper-body joint positions, semantic segmentation with a fully convolutional network was exploited. From acquired RGB images, joint heatmaps accurately estimate the coordinates of the location of each joint. The network architecture was designed to learn and detect the locations of joints via the sequential prediction processing method. Our proposed method was tested and validated for efficient estimation of the human upper-body pose. The obtained results reveal the potential of a simple RGB camera sensor for human pose estimation applications.

3D 피트니스 인체 포즈 추정을 위한 MoveNet의 확장

박병준,윤경로 한국방송∙미디어공학회 2024 방송공학회논문지 Vol.29 No.1

Human pose estimation from RGB images without using other sensors is an important point for cost reduction and usability,and it has become increasingly popular in recent years due to the need in various fields such as HCI, robotics, video analytics,and metaverse. There have been many researches and development attempts on pose estimation, but 3D human pose estimationfrom a monocular image is experiencing difficulties in research due to depth ambiguity, object occlusion, background disorder, andlack of training data. Recent papers design models with a focus on occlusion, but many existing 3D pose training data do notinclude occlusion information. In this paper, we propose a 3D human pose estimator that can infer quickly from data withoutocclusion using RGB images. Based on MoveNet, which is a 2D human body pose estimator, it was transformed to be able toestimate 3D pose. In order to optimize the hyperparameters for 3D pose estimation, we conducted an experiment to diversify thehyperparameters, and compared the performance with the state-of-the-art papers using a monocular RGB image.

Empirical Comparison of Deep Learning Networks on Backbone Method of Human Pose Estimation

( Beanbonyka Rim ),( Junseob Kim ),( Yoo-joo Choi ),( Min Hong ) 한국인터넷정보학회 2020 인터넷정보학회논문지 Vol.21 No.5

Accurate estimation of human pose relies on backbone method in which its role is to extract feature map. Up to dated, the method of backbone feature extraction is conducted by the plain convolutional neural networks named by CNN and the residual neural networks named by Resnet, both of which have various architectures and performances. The CNN family network such as VGG which is well-known as a multiple stacked hidden layers architecture of deep learning methods, is base and simple while Resnet which is a bottleneck layers architecture yields fewer parameters and outperform. They have achieved inspired results as a backbone network in human pose estimation. However, they were used then followed by different pose estimation networks named by pose parsing module. Therefore, in this paper, we present a comparison between the plain CNN family network (VGG) and bottleneck network (Resnet) as a backbone method in the same pose parsing module. We investigate their performances such as number of parameters, loss score, precision and recall. We experiment them in the bottom-up method of human pose estimation system by adapted the pose parsing module of openpose. Our experimental results show that the backbone method using VGG network outperforms the Resent network with fewer parameter, lower loss score and higher accuracy of precision and recall.

단일 이미지에 기반을 둔 사람의 포즈 추정에 대한 연구 동향

조정찬 한국차세대컴퓨팅학회 2019 한국차세대컴퓨팅학회 논문지 Vol.15 No.5

With the recent development of deep learning technology, remarkable achievements have been made in many research areas of computer vision. Deep learning has also made dramatic improvement in two-dimensional or three-dimensional human pose estimation based on a single image, and many researchers have been expanding the scope of this problem. The human pose estimation is one of the most important research fields because there are various applications, especially it is a key factor in understanding the behavior, state, and intention of people in image or video analysis. Based on this background, this paper surveys research trends in estimating human poses based on a single image. Because there are various research results for robust and accurate human pose estimation, this paper introduces them in two separated subsections: 2D human pose estimation and 3D human pose estimation. Moreover, this paper summarizes famous data sets used in this field and introduces various studies which utilize human poses to solve their own problem. 최근 딥러닝 기술이 발전함에 따라 많은 컴퓨터 비전 연구 분야에서 주목할 만한 성과들이 지속적으로 나오고 있다. 단일 이미지를 기반으로 사람의 2차원 및 3차원 포즈를 추정하는 연구에서도 비약적인 성능향상을 보여주고 있으며, 많은 연구자들이 문제의 범위를 확장하며 활발한 연구 활동을 진행하고 있다. 사람의 포즈 추정은 다양한 응용 분야가 존재하고, 특히 이미지나 비디오 분석에서 사람의 포즈는 행동 및 상태, 의도 파악을 위한 핵심 요소가 되기 때문에 상당히 중요한 연구 분야이다. 이러한 배경에 따라 본 논문은 단일 이미지를 기반으로 한 사람의 포즈 추정 기술에 대한 연구 동향을 살펴보고자 한다. 강인하고 정확한 문제 해결을 위해 다양한 연구 활동 결과가 존재한다는 점에서 본 논문에서는 사람의 포즈 추정 연구를 2차원 및 3차원 포즈 추정에 대해서 나누어 살펴보고자 한다. 끝으로 연구에 필요한 데이터 세트 및 사람의 포즈 추정 기술을 적용하는 다양한 연구 사례를 살펴볼 것이다.

인공지능 : 자세 예측을 이용한 효과적인 자세 기반 감정 동작 인식

김진옥 ( Jin Ok Kim ) 한국정보처리학회 2013 정보처리학회논문지. 소프트웨어 및 데이터 공학 Vol.2 No.3

Early researches in human action recognition have focused on tracking and classifying articulated body motions. Such methods required accurate segmentation of body parts, which is a sticky task, particularly under realistic imaging conditions. Recent trends of work have become popular towards the use of more abstract and low-level appearance features such as spatio-temporal interest points. Given the great progress in pose estimation over the past few years, redefined views about pose-based approach are needed. This paper addresses the issues of whether it is sufficient to train a classifier only on low-level appearance features in appearance approach and proposes effective pose-based approach with pose estimation for emotional action recognition. In order for these questions to be solved, we compare the performance of pose-based, appearance-based and its combination-based features respectively with respect to scenario of various emotional action recognition. The experiment results show that pose-based features outperform low-level appearance-based approach of features, even when heavily spoiled by noise, suggesting that pose-based approach with pose estimation is beneficial for the emotional action recognition.

아웃페인팅 기반 반려동물 자세 추정에 관한 예비 연구

이규빈,이영찬,유원상 한국융합신호처리학회 2023 융합신호처리학회 논문지 (JISPS) Vol.24 No.1

In recent years, there has been a growing interest in deep learning-based animal pose estimation, especially in the areas of animal behavior analysis and healthcare. However, existing animal pose estimation techniques do not perform well when body parts are occluded or not present. In particular, the occlusion of dog tail or ear might lead to a significant degradation of performance in pet behavior and emotion recognition. In this paper, to solve this intractable problem, we propose a simple yet novel framework for pet pose estimation where pet pose is predicted on an outpainted image where some body parts hidden outside the input image are reconstructed by the image inpainting network preceding the pose estimation network, and we performed a preliminary study to test the feasibility of the proposed approach. We assessed CE-GAN and BAT-Fill for image outpainting, and evaluated SimpleBaseline for pet pose estimation. Our experimental results show that pet pose estimation on outpainted images generated using BAT-Fill outperforms the existing methods of pose estimation on outpainting-less input image.