입술 영역의 움직임과 밝기 변화를 이용한 음성구간 검출 알고리즘 개발

다국어 초록 (Multilingual Abstract)

Voice activity detection (VAD) is generally conducted by extracting features from the acoustic signal and a decision rule. The performance of such VAD algorithms driven by the input acoustic signal highly depends on the acoustic noise. When video signals are available as well, the performance of VAD can be enhanced by using the visual information which is not affected by the acoustic noise. Previous visual VAD algorithms usually use single visual feature to detect the lip activity, such as active appearance models, optical flow or intensity variation. Based on the analysis of the weakness of each feature, we propose to combine intensity change measure and the optical flow in the mouth region, which can compensate for each other’s weakness. In order to minimize the computational complexity, we develop simple measures that avoid statistical estimation or modeling. Specifically, the optical flow is the averaged motion vector of some grid regions and the intensity variation is detected by simple thresholding. To extract the mouth region, we propose a simple algorithm which first detects two eyes and uses the profile of intensity to detect the center of mouth. Experiments show that the proposed combination of two simple measures show higher detection rates for the given false positive rate than the methods that use a single feature.

참고문헌 (Reference)

1 H. Yehia, "uantitative association of vocal-tract and facial behavior" 26 (26): 23-43, 1998

2 P. Liu, "Voice activity detection using visual information" 609-612, 2004

3 G. Kim, "Voice activity detection using phase vector in microphone array" 43 (43): 783-784, 2007

4 A. Aubrey, "Visual voice activity detection with optical flow" 4 (4): 463-472, 2010

5 R. Navarathna, "Visual voice activity detection using frontal versus profile views" 2011

6 S. Siatras, "Visual lip activity detection and speaker detection using mouth region intensities" 19 (19): 133-137, 2009

7 L. Armani, "Use of a CSP-based voice activity detector for distant-talking ASR" 2003

8 A. Aubrey, "Two novel visual voice activity detectors based on appearance models and retinal filtering" 2007

9 S. F. Boll, "Suppression of acoustic noise in speech using spectral subtraction" ASSP-27ASS (ASSP-27ASS): 113-120, 1979

10 D. Sun, "Secrets of optical flow estimation and their principles" 2432-2439, 2010

11 B.-F. Wu, "Robust endpoint detection algorithm based on the adaptive band-partitioning spectral entropy in adverse environments" 13 (13): 762-775, 2005

12 P. Viola, "Robust Real-time Object Detection" 2001

13 S. Tamura, "Multi-modal speech recognition using optical-flow analysis for lip images" 36 : 117-124, 2004

14 G. Fanelli, "Hough Transform-based Mouth Localization for Audio-Visual Speech Recognition" 2009

15 M. Hoffman, "GSC-based spatial voice activity detection for enhanced speech coding in the presence of competing speech" 9 (9): 175-179, 2001

16 R. Navarathna, "Cascading appearance-based features for visual voice activity detection" 2010

17 B. Lucas, "An iterative image registration technique with an application to stereo vision" 674-679, 1981

18 L. F. Lamel, "An improved endpoint detector for isolated word recognition" 29ASSP-29 (29ASSP-29): 777-785, 1981

19 E. Skodras, "An Unconstrained Method for Lip Detection in Color Images" 2011

20 T. Cootes, "Active appearance models" 23 (23): 681-685, 2001

21 J. Sohn, "A statistical model-based voice activity detection" 6 (6): 1-3, 1999

22 Y. Freund, "A decision-theoretic generalization of on-line learning and an application to boosting In Computational Learning Theory" Springer-Verlag 23-37, 1995

23 X. Liu, "A Lip Contour Extraction Method Using Localized Active Contour Model with Automatic Parameter Selection" 2010