Optical Flow Hand Tracking and Active Contour Hand Shape Features for Continuous Sign Language Recognition with Artificial Neural Networks

다국어 초록 (Multilingual Abstract)

To extract hand tracks and hand shape features from continuous sign language videos for gesture classification using backpropagation neural network. Horn Schunck optical flow (HSOF) extracts tracking features and Active Contours (AC) extract shape features. A feature matrix characterizes the signs in continuous sign videos. A neural network object with backpropagation training algorithm classifies the signs into various words sequences in digital format. Digital word sequences are translated into text with matching and the suiting text is voice translated using windows application programmable interface (Win-API). Ten signers, each doing sentences having 30 words long tests the performance of the algorithm by computing word matching score (WMS). The WMS is varying between 88 and 91 percent when executed on different cross platforms on various processors such as Windows8 with Inteli3, Windows8.1 with inteli3 and windows10 with inteli3 running MATLAB13(a).

목차 (Table of Contents)

• Abstract
• 1. Introduction
• 2. Tracking with Optical Flow – Revisit
• 3. Shape Segmentation with Level Sets – Revisit
• 4. Continuous Sign Language Recognizer – Proposed Model
• 5. Results and Discussion
• 6. Conclusion
• References