Efficient Head Pose Determination and Its Application to Face Recognition on Multi-Pose Face DB

Jun Lee,Jeong-Sik Park,Gil-Jin Jang,Yong-Ho Seo 보안공학연구지원센터 2016 International Journal of Multimedia and Ubiquitous Vol.11 No.2

Face recognition is a well-known approach for identity recognition. Variation in head pose is a main factor that interferes with face recognition systems. This paper proposes an efficient head pose determination method and its application to face recognition on a multi-pose face DB in order to solve the pose variation-related problem. The first step is to detect a facial region using Adaboost. Next, after undergoing preprocessing on the detected face, a mask is placed to cover it. At the detected facial region, the pose is determined by relations of the position of the centroid points of the eyes and lip regions detected by using ellipse-fitting method. Finally, face recognition is conducted by applying template matching between a set of facial images in multi-pose face DB pertinent to the determined head pose and the input face image. In experiments, the proposed approach outperformed the conventional PCA-based face recognition approach depending on a single-pose face DB.

트랜스포머 기반의 다중 시점 3차원 인체자세추정

최승욱(Seoung Wook Choi),이진영(Jin Young Lee),김계영(Gye Young Kim) 한국스마트미디어학회 2023 스마트미디어저널 Vol.12 No.11

3차원 인체자세추정은 스포츠, 동작인식, 영상매체의 특수효과 등의 분야에서 널리 활용되고 있는 기술이다. 이를 위한 여러 방법들 중 다중 시점 3차원 인체자세추정은 현실의 복잡한 환경에서도 정밀한 추정을 하기 위해 필수적인 방법이다. 하지만 기존 다중 시점 3차원 인체자세추정 모델들은 3차원 특징 맵을 사용함에 따라 시간 복잡도가 높은 단점이 있다. 본 논문은 계산 복잡도가 적은 트랜스포머 기반 기존 단안 시점 다중 프레임 모델을 다중 시점에 대한 3차원 인체자세추정으로 확장하는 방법을 제안한다. 다중 시점으로 확장하기 위하여 먼저 2차원 인체자세 검출자 CPN(Cascaded Pyramid Network)을 활용하여 획득한 4개 시점의 17가지 관절에 대한 2차원 관절좌표를 연결한 8차원 관절좌표를 생성한다. 그 다음 이들을 패치 임베딩 한 뒤 17×32 데이터로 변환하여 트랜스포머 모델에 입력한다. 마지막으로, 인체자세를 출력하는 MLP(Multi-Layer Perceptron) 블록을 매 반복 마다 사용한다. 이를 통해 4개 시점에 대한 3차원 인체자세추정을 동시에 수정한다. 입력 프레임 길이 27을 사용한 Zheng[5]의 방법과 비교했을 때 제안한 방법의 모델 매개변수의 수는 48.9%, MPJPE(Mean Per Joint Position Error)는 20.6mm(43.8%) 감소했으며, 학습 횟수 당 평균 학습 소요 시간은 20배 이상 빠르다. The technology of Three-dimensional human posture estimation is used in sports, motion recognition, and special effects of video media. Among various methods for this, multi-view 3D human pose estimation is essential for precise estimation even in complex real-world environments. But Existing models for multi-view 3D human posture estimation have the disadvantage of high order of time complexity as they use 3D feature maps. This paper proposes a method to extend an existing monocular viewpoint multi-frame model based on Transformer with lower time complexity to 3D human posture estimation for multi-viewpoints. To expand to multi-viewpoints our proposed method first generates an 8-dimensional joint coordinate that connects 2-dimensional joint coordinates for 17 joints at 4-vieiwpoints acquired using the 2-dimensional human posture detector, CPN(Cascaded Pyramid Network). This paper then converts them into 17×32 data with patch embedding, and enters the data into a transformer model, finally. Consequently, the MLP(Multi-Layer Perceptron) block that outputs the 3D-human posture simultaneously updates the 3D human posture estimation for 4-viewpoints at every iteration. Compared to Zheng[5]'s method the number of model parameters of the proposed method was 48.9%, MPJPE(Mean Per Joint Position Error) was reduced by 20.6 mm (43.8%) and the average learning time per epoch was more than 20 times faster.

A Multi-Stage Convolution Machine with Scaling and Dilation for Human Pose Estimation

( Yali Nie ),( Jaehwan Lee ),( Sook Yoon ),( Dong Sun Park ) 한국인터넷정보학회 2019 KSII Transactions on Internet and Information Syst Vol.13 No.6

Vision-based Human Pose Estimation has been considered as one of challenging research subjects due to problems including confounding background clutter, diversity of human appearances and illumination changes in scenes. To tackle these problems, we propose to use a new multi-stage convolution machine for estimating human pose. To provide better heatmap prediction of body joints, the proposed machine repeatedly produces multiple predictions according to stages with receptive field large enough for learning the long-range spatial relationship. And stages are composed of various modules according to their strategic purposes. Pyramid stacking module and dilation module are used to handle problem of human pose at multiple scales. Their multi-scale information from different receptive fields are fused with concatenation, which can catch more contextual information from different features. And spatial and channel information of a given input are converted to gating factors by squeezing the feature maps to a single numeric value based on its importance in order to give each of the network channels different weights. Compared with other ConvNet-based architectures, we demonstrated that our proposed architecture achieved higher accuracy on experiments using standard benchmarks of LSP and MPII pose datasets.

Multi-sensor-based Target Pose Estimation for Autonomous Precision Perching of Nano Aerial Vehicles

Truong-Dong Do,Nguyen Xuan-Mung,Ngoc-Phi Nguyen,Ji-Won Lee,Yong-Seok Lee,Seok-Tae Lee,Sung-Kyung Hong 제어로봇시스템학회 2022 제어로봇시스템학회 국제학술대회 논문집 Vol.2022 No.11

Nano Aerial Vehicles have become widely used for a variety of complex missions due to their mobility and the ability to access hard-to-reach areas. In most cases, these tasks require the vehicles to land on ground targets or perch on platforms mounted on diverse surfaces. Considering the surface the vehicle will reach, controlling perching is obviously a challenging task. Besides, the reliability of target position and direction estimation has a significant impact on perching performance. In this paper, a multi-sensor-based target pose estimation for autonomous precision perching of nano drones is proposed. First, the perching target, a cube cage containing a small marker inside a larger one, is designed to enhance pose estimation capability at a wide range of distances. Second, we constructed a nano drone with an upward monocular camera and a 5-direction multi-ranger deck. Next, the flying vehicle’s pose toward the perching target is calculated, followed by a Kalman filter for filtering and estimating the missing data. Finally, we introduced an algorithm to merge the pose data from multiple sensors when drones approach close to the target. Real measurements are conducted on the testbed. The experimental results demonstrated the utility and potential of the adopted approach with millimeter-level precision.

Pose Estimation with Binarized Multi-Scale Module

Choi, Yong-Gyun,Lee, Sukho The Institute of Internet 2018 International journal of advanced smart convergenc Vol.7 No.2

In this paper, we propose a binarized multi-scale module to accelerate the speed of the pose estimating deep neural network. Recently, deep learning is also used for fine-tuned tasks such as pose estimation. One of the best performing pose estimation methods is based on the usage of two neural networks where one computes the heat maps of the body parts and the other computes the part affinity fields between the body parts. However, the convolution filtering with a large kernel filter takes much time in this model. To accelerate the speed in this model, we propose to change the large kernel filters with binarized multi-scale modules. The large receptive field is captured by the multi-scale structure which also prevents the dropdown of the accuracy in the binarized module. The computation cost and number of parameters becomes small which results in increased speed performance.

다시점 객체 공분할을 이용한 2D-3D 물체 자세 추정

김성흠,복윤수,권인소 한국로봇학회 2017 로봇학회 논문지 Vol.12 No.1

We present a region-based approach for accurate pose estimation of small mechanical components. Our algorithm consists of two key phases: Multi-view object co-segmentation and pose estimation. In the first phase, we explain an automatic method to extract binary masks of a target object captured from multiple viewpoints. For initialization, we assume the target object is bounded by the convex volume of interest defined by a few user inputs. The co-segmented target object shares the same geometric representation in space, and has distinctive color models from those of the backgrounds. In the second phase, we retrieve a 3D model instance with correct upright orientation, and estimate a relative pose of the object observed from images. Our energy function, combining region and boundary terms for the proposed measures, maximizes the overlapping regions and boundaries between the multi-view co-segmentations and projected masks of the reference model. Based on high-quality co-segmentations consistent across all different viewpoints, our final results are accurate model indices and pose parameters of the extracted object. We demonstrate the effectiveness of the proposed method using various examples.

Pose Estimation with Binarized Multi-Scale Module

최용균,이석호 한국인터넷방송통신학회 2018 Journal of Advanced Smart Convergence Vol.7 No.2

In this paper, we propose a binarized multi-scale module to accelerate the speed of the pose estimating deep neural network. Recently, deep learning is also used for fine-tuned tasks such as pose estimation. One of the best performing pose estimation methods is based on the usage of two neural networks where one computes the heat maps of the body parts and the other computes the part affinity fields between the body parts. However, the convolution filtering with a large kernel filter takes much time in this model. To accelerate the speed in this model, we propose to change the large kernel filters with binarized multi-scale modules. The large receptive field is captured by the multi-scale structure which also prevents the dropdown of the accuracy in the binarized module. The computation cost and number of parameters becomes small which results in increased speed performance.

Pose Estimation with Binarized Multi-Scale Module

Yong-Gyun Choi,Sukho Lee 한국인터넷방송통신학회 2018 Journal of Advanced Smart Convergence Vol.7 No.2

In this paper, we propose a binarized multi-scale module to accelerate the speed of the pose estimating deep neural network. Recently, deep learning is also used for fine-tuned tasks such as pose estimation. One of the best performing pose estimation methods is based on the usage of two neural networks where one computes the heat maps of the body parts and the other computes the part affinity fields between the body parts. However, the convolution filtering with a large kernel filter takes much time in this model. To accelerate the speed in this model, we propose to change the large kernel filters with binarized multi-scale modules. The large receptive field is captured by the multi-scale structure which also prevents the dropdown of the accuracy in the binarized module. The computation cost and number of parameters becomes small which results in increased speed performance.

A Novel Multi-view Face Detection Method Based on Improved Real Adaboost Algorithm

( Wenkai Xu ),( Eung-joo Lee ) 한국인터넷정보학회 2013 KSII Transactions on Internet and Information Syst Vol.7 No.11

Multi-view face detection has become an active area for research in the last few years. In this paper, a novel multi-view human face detection algorithm based on improved real Adaboost is presented. Real Adaboost algorithm is improved by weighted combination of weak classifiers and the approximately best combination coefficients are obtained. After that, we proved that the function of sample weight adjusting method and weak classifier training method is to guarantee the independence of weak classifiers. A coarse-to-fine hierarchical face detector combining the high efficiency of Haar feature with pose estimation phase based on our real Adaboost algorithm is proposed. This algorithm reduces training time cost greatly compared with classical real Adaboost algorithm. In addition, it speeds up strong classifier converging and reduces the number of weak classifiers. For frontal face detection, the experiments on MIT+CMU frontal face test set result a 96.4% correct rate with 528 false alarms; for multi-view face in real time test set result a 94.7 % correct rate. The experimental results verified the effectiveness of the proposed approach.

Marker-based Multi-camera Real-time Dense Point Cloud Reconstruction in Orchards

정육 ( Yu Zheng ),김우영 ( Woo-young Kim ),이경환 ( Kyeong-hwan Lee ) 한국농업기계학회 2022 한국농업기계학회 학술발표논문집 Vol.27 No.1

Phenotype analysis and digitization are essential but labor-intensive activities in orchards. Although current multi-view 3D dense reconstructions are able to obtain 3D models of fruit trees, specific tasks can not be done in real time due to the high computational effort and time-consuming cost. In this research, we developed a new technique to achieve fast real-time 3D reconstruction in orchards. The 3D point cloud frames were collected from a multi-camera system using a RGB-depth camera. The multi-camera system determined mutual poses by spatial matching algorithms based on the 3D feature values extracted from point clouds. The 3D model of the fruit tree was rebuilt using reconstruction algorithms, including pose map creation, marker closed-loop edge detection, and closed-loop optimization methods. The developed 3D reconstruction algorithm maped the actual spatial information of fruit trees into 3D point cloud space by point cloud frames assembling. The results obtained from the proposed method were in good agreement with the manual measurements. The determination coefficient (R2) of 0.996 and root mean square error (RMSE) value of 9.55 mm was obtained in the 3D reconstruction model. The experimental results quantitatively showed the high accuracy and robustness of our method. In this study, we proposed a method that combines multi-camera systems and real-time 3D reconstruction to achieve a reliable 3D model reconstruction of fruit trees in orchards. This method will be applicable for accurate phenotype analysis and digitization.