Electroencephalography-based imagined speech recognition using deep long short-term memory network

다국어 초록 (Multilingual Abstract)

This article proposes a subject-independent application of brain–computer interfacing (BCI). A 32-channel Electroencephalography (EEG) device is used to measure imagined speech (SI) of four words (sos, stop, medicine, washroom) and one phrase (come-here) across 13 subjects. A deep long short-term memory (LSTM) network has been adopted to recognize the above signals in seven EEG frequency bands individually in nine major regions of the brain. The results show a maximum accuracy of 73.56% and a network prediction time (NPT) of 0.14 s which are superior to other state-of-the-art techniques in the literature. Our analysis reveals that the alpha band can recognize SI better than other EEG frequencies. To reinforce our findings, the above work has been compared by models based on the gated recurrent unit (GRU), convolutional neural network (CNN), and six conventional classifiers. The results show that the LSTM model has 46.86% more average accuracy in the alpha band and 74.54% less average NPT than CNN. The maximum accuracy of GRU was 8.34% less than the LSTM network. Deep networks performed better than traditional classifiers.

참고문헌 (Reference)

1 K. Khanna, "“The locked-in syn drome”: Can it be unlocked?" 2 (2): 96-99, 2011

2 S. Martin, "Word pair classification during imagined speech using direct brain recordings" 6 : 25803-, 2016

3 X. Glorot, "Understanding the difficulty of training deep feedforward neural networks" 249-256, 2010

4 P. Agarwal, "Transforming imagined thoughts into speech using a covariance-based subset selection method" 59 (59): 180-183, 2021

5 G. H. Klem, "The ten twenty electrode system of the international federation. The international federation of clinical neurophysiology" 52 : 3-6, 1999

6 E. F. González-Castañeda, "Sonification and textification:Proposing methods for classifying unspoken words from EEG signals" 37 : 82-91, 2017

7 C. S. Dasalla, "Single-trial classification of vowel speech imagery using common spatial patterns" 22 (22): 1334-1339, 2009

8 P. Agarwal, "Silent speech classification based upon various feature extraction methods" 16-20, 2020

9 Sandeep Kumar, "Real‐time implementation and performance evaluation of speech classifiers in speech analysis‐synthesis" 한국전자통신연구원 43 (43): 82-94, 2021

10 J. Hazarika, "Neural modulation in action video game players during inhibitory control function: An EEG study using discrete wavelet transform" 45 : 144-150, 2018

11 Dipti Pawar ; Sudhir Dhage, "Multiclass covert speech classifi cation using extreme learning machine" 대한의용생체공학회 10 (10): 217-226, 2020

12 M. N. I. Qureshi, "Multiclass classification of word imagination speech with hybrid connectivity features" 65 (65): 2168-2177, 2018

13 S. Hochreiter, "Long Short-Term Memory" 9 (9): 1735-1789, 1997

14 C. H. Nguyen, "Inferring imagined speech using EEG signals: A new approach using Riemannian manifold features" 15 (15): 016002-, 2017

15 A. A. Torres-García, "Implementing a fuzzy inference system in a multi-objective EEG channel selection model for imagined speech classification" 59 : 1-12, 2016

16 K. Brigham, "Imagined speech classification with EEG signals for silent communication: A preliminary investigation into synthetic telepathy" 1-4, 2010

17 M. D’Zamura, "Human-computer interaction. New trends Vol. 5610" Springer 40-48, 2009

18 P. Saha, "Hierarchical deep feature learning for decoding imagined speech from EEG" 10019-10020, 2019

19 T. K. Reddy, "HJB-equation-based optimal learning scheme for neural networks with applications in brain-computer interface" 4 (4): 159-170, 2020

20 J. M. Lilly, "Generalized Morse wavelets as a superfamily of analytic wavelets" 60 (60): 6036-6041, 2012

21 S. Wellington, "Fourteen-channel EEG with Imagined Speech (FEIS) dataset, v1.0" University of Edinburgh 3554128-, 2019

22 C. Ju, "Federated transfer learning for EEG signal classification" 3040-3045, 2020

23 A. M. Saxe, "Exact solutions to the nonlinear dynamics of learning in deep linear neural networks" 2013

24 P. Kumar, "Envisioned speech recognition using EEG sensors" 22 : 185-199, 2018

25 Prabhakar Agarwal, "Electroencephalography based imagined alphabets classification using spatial and time‐domain features" Wiley 32 (32): 111-122, 2021

26 A. Porbadnigk, "EEG-based speech recognition- impact of temporal effects" 376-381, 2009

27 P. Kaushik, "EEG-based age and gender prediction using deep BLSTM-LSTM network model" 19 (19): 2634-2641, 2019

28 S. Siuly, "EEG signal analysis and classification: Techniques and applications" Springer 2016

29 P. Saha, "Deep learning the EEG manifold for phonological categorization from active thoughts" 2762-2766, 2019

30 S. Kellis, "Decoding spoken words using local field potentials recorded from the cortical surface" 7 (7): 056007-, 2010

31 D. Dash, "Decoding imagined and spoken phrases from non-invasive neural (MEG) signals" 14 : 290-, 2020

32 L. Marple, "Computing the discrete-time “analytic” signal via FFT" 47 (47): 2600-2603, 1999

33 S. Zhao, "Classifying phonological categories in imagined and articulated speech" 992-996, 2015

34 M.-O. Tamm, "Classification of vowels from imagined speech with convolutional neural networks" 9 (9): 46-, 2020

35 E. T. Esfahani, "Classification of primitive shapes using brain-computer interfaces" 44 (44): 1011-1019, 2012

36 C. Cooney, "Classification of imagined spoken word-pairs using convolutional neural networks" 338-343, 2019

37 P. Kant, "CWT based transfer learning for motor imagery classification for brain computer interfaces" 345 : 108886-, 2020

38 R. A. Ramadan, "Brain computer interface:Control signals review" 223 : 26-44, 2017

39 이미란 ; 류재환 ; 김덕환, "Automated epileptic seizure waveform detection method based on the feature of the mean slope of wavelet coefficient counts using a hidden Markov model and EEG signals" 한국전자통신연구원 42 (42): 217-229, 2020

40 A. M. Choudhari, "An electrooculography based human machine interface for wheelchair control," 39 (39): 673-685, 2019

41 O. Özdenizci, "Adversarial deep learning in EEG biometrics" 26 (26): 710-714, 2019

42 W. He, "A wireless BCI and BMI system for wearable robots" 46 (46): 936-946, 2016

43 A. Khosla, "A comparative analysis of signal processing and classification methods for different applications based on EEG signals" 40 (40): 649-690, 2020

44 Sandeep Kumar, "A CNN based graphical user interface controlled by imagined movements" Springer Science and Business Media LLC 2021

45 Ki-Hong Kim ; Hong Kee Kim ; Jong-Sung Kim ; Wookho Son ; 이수영, "A Biosignal-Based Human Interface Controlling a Power-Wheelchair for People with Motor Disabilities" 한국전자통신연구원 28 (28): 111-114, 2006