Robust Multithreaded Object Tracker through Occlusions for Spatial Augmented Reality

다국어 초록 (Multilingual Abstract)

A spatial augmented reality (SAR) system enables a virtual image to be projected onto the surface of a real‐world object and the user to intuitively control the image using a tangible interface. However, occlusions frequently occur, such as a sudden change in the lighting environment or the generation of obstacles. We propose a robust object tracker based on a multithreaded system, which can track an object robustly through occlusions. Our multithreaded tracker is divided into two threads: the detection thread detects distinctive features in a frame‐to‐frame manner, and the tracking thread tracks features periodically using an optical‐flow‐based tracking method. Consequently, although the speed of the detection thread is considerably slow, we achieve real‐time performance owing to the multithreaded configuration. Moreover, the proposed outlier filtering automatically updates a random sample consensus distance threshold for eliminating outliers according to environmental changes. Experimental results show that our approach tracks an object robustly in real‐time in an SAR environment where there are frequent occlusions occurring from augmented projection images.

참고문헌 (Reference)

1 X. Wang, "The Use of Object Tracking in Visual SLAM" 850-853, 2017

2 J.M. Prewitt, "The Analysis of Cell Images" 128 (128): 1035-1053, 1966

3 H. Bay, "Speeded-Up Robust Features (SURF)" 3951 : 404-417, 2006

4 Y. Xiong, "Sequential Image Stitching for Nobile Panoramas" 2009

5 J.H. Cha, "Seamless and Fast Panoramic Image Stitching" 29-30, 2012

6 최진우, "Robust Multi-person Tracking for Real-Time Intelligent Video Surveillance" 한국전자통신연구원 37 (37): 551-561, 2015

7 L. Cheng, "Remote Sensing Image Matching by Integrating Affine Invariant Feature Extraction and RANSAC" 38 (38): 1023-1032, 2012

8 M.A. Fischler, "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography" 24 (24): 381-395, 1981

9 J.-Y. Bouguet, "Pyramidal Implementation of the Affine Lucas Kanade Feature Tracker Description of the Algorithm" 5 : 1-10, 2001

10 F. Zhang, "Parallax-Tolerant Image Stitching" 3262-3269, 2014

11 W. Doyle, "Operations Useful for Similarity-Invariant Pattern Recognition" 9 (9): 259-267, 1962

12 E. Rublee, "ORB: An Efficient Alternative to SIFT or SURF" 2564-2571, 2011

13 R. Mur-Artal, "ORB- SLAM: A Versatile and Accurate Monocular SLAM System" 31 (31): 1147-1163, 2015

14 T. Lee, "Multithreaded Hybrid Feature Tracking for Markerless Augmented Reality" 15 (15): 355-368, 2009

15 A. Lee, "Multi-threaded Tracker with Outlier Filtering for Spatial Augmented Reality" 494-495, 2015

16 A. Davison, "MonoSLAM: Real-Time Single Camera SLAM" 29 (29): 1052-1067, 2007

17 W.H. Tsai, "Moment-Preserving Thresholding: A New Approach" 29 (29): 377-393, 1985

18 H. Benko, "MirageTable: Freehand Interaction on a Projected Augmented Reality Tabletop" 199-208, 2012

19 M. Fornasier, "Iterative Thresholding Algorithms" 25 (25): 187-208, 2008

20 A. Lee, "Interactive Design of Planar Curves Based on Spatial Augmented Reality" 53-54, 2013

21 J.H. Lee et al., "FRC Based Augment Reality for Aiding Cooperative Activities" 294-295, 2013

22 R. Ziola, "Examining Interaction with General-Purpose Object Recognition in LEGO OASIS" 65-68, 2011

23 D. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints" 60 (60): 91-110, 2004

24 이아현, "Data-Driven Kinematic Control for Robotic Spatial Augmented Reality System with Loose Kinematic Specifications" 한국전자통신연구원 38 (38): 337-346, 2016

25 J. Weng, "Camera Calibration with Distortion Models and Accuracy Evaluation" 14 (14): 965-980, 1992

26 J.-H. Lee et al., "Calibration Issues in FRC: Camera, Projector, Kinematics Based Hybrid Approach" 218-219, 2012

27 M. Brown, "Automatic Panoramic Image Stitching Using Invariant Features" 74 (74): 59-73, 2007

28 이주행, "An Analytic Solution to Projector Pose Estimation Problem" 한국전자통신연구원 34 (34): 978-981, 2012

29 R. Tsai, "A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off- the-Shelf TV Cameras and Lenses" 3 (3): 323-344, 1987

30 N. Otsu, "A Threshold Selection Method from Gray-Level Histograms" 9 : 62-66, 1975

31 Z. Zhang, "A Flexible New Technique for Camera Calibration" 22 (22): 1330-1334, 2000