사용자의 신체적 특징과 뇌파 집중 지수를 이용한 가상 모니터 개념의 NUI/NUX

전창현 ( Chang-hyun Jeon ),안소영 ( So-young Ahn ),신동일 ( Dong-il Shin ),신동규 ( Dong-kyoo Shin ) 한국인터넷정보학회 2015 인터넷정보학회논문지 Vol.16 No.6

Human-Computer Interaction(HCI)에 대한 관심도가 높이지면서, HCI에 대한 연구도 활발히 진행되고 있다. 이와 더불어 사용자의 몸짓이나 음성을 이용하는 Natural User Interface/Natural User eXperience(NUI/NUX)에 대한 연구도 활발히 진행되고 있다. NUI/NUX의 경우, 제스처 인식이나 음성 인식 등의 인식 알고리즘이 필요하다. 하지만 이러한 인식 알고리즘은 전처리, 정규화, 특징 추출과 같은 단계를 거쳐야하기 때문에 구현이 복잡하고, 트레이닝에 많은 시간을 투자해야 한다는 단점이 있다. 최근에는 NUI/NUX 개발 도구로 Microsoft 사의 Kinect가 개발되어 개발자와 일반인들에게 많은 관심을 받고 있고, 이를 이용한 다양한 연구가 진행 중에 있다. 본 저자들의 이전 연구에서도 사용자의 신체적 특징을 이용하여 뛰어난 직관성을 가진 핸드 마우스를 구현하였다. 하지만 마우스의 움직임이 부자연스럽고 정확도가 낮아 사용자가 사용하기 다소 어려웠다는 단점이 있다. 본 연구에서는 Kinect를 통해 사용자의 신체적 특징을 실시간으로 추출하고, 이를 이용해 가상 모니터라는 새로운 개념을 추가한 핸드 마우스 인터페이스를 설계하고 구현하였다. 가상 모니터는 사용자의 손으로 마우스를 제어할 수 있는 가상의 공간을 의미한다. 이를 통해 가상 모니터 상의 손의 좌표를 실제 모니터 상의 좌표로 정확하게 매핑(mapping)이 가능하다. 가상 모니터를 사용함으로써 이전 연구의 장점인 직관성을 유지하고, 단점인 정확도를 높일 수 있다. 추가적으로 뇌파 집중 지표를 이용해 사용자의 불필요한 행동을 인식하여 핸드 마우스 인터페이스의 정확도를 높였다. 제안하는 핸드 마우스의 직관성과 정확성을 평가하기 위하여 10대부터 50대까지 50명에게 실험을 하였다. 직관성 실험 결과로 84%가 1분 이내에 사용방법을 터득하였다. 또한 동일한 피실험자에게 일반적인 마우스 기능(드래그, 클릭, 더블클릭)에 대해 정확성 실험을 한 결과로 드래그 80.9%, 클릭 80%, 더블 클릭 76.7%의 정확성을 보였다. 실험 결과를 통해 제안하는 핸드 마우스 인터페이스의 직관성과 정확성을 확인하였으며, 미래에 손으로 시스템이나 소프트웨어를 제어하는 인터페이스의 좋은 예시가 될 것으로 기대된다. As growing interest in Human-Computer Interaction(HCI), research on HCI has been actively conducted. Also with that, research on Natural User Interface/Natural User eXperience(NUI/NUX) that uses user`s gesture and voice has been actively conducted. In case of NUI/NUX, it needs recognition algorithm such as gesture recognition or voice recognition. However these recognition algorithms have weakness because their implementation is complex and a lot of time are needed in training because they have to go through steps including preprocessing, normalization, feature extraction. Recently, Kinect is launched by Microsoft as NUI/NUX development tool which attracts people`s attention, and studies using Kinect has been conducted. The authors of this paper implemented hand-mouse interface with outstanding intuitiveness using the physical features of a user in a previous study. However, there are weaknesses such as unnatural movement of mouse and low accuracy of mouse functions. In this study, we designed and implemented a hand mouse interface which introduce a new concept called `Virtual monitor` extracting user`s physical features through Kinect in real-time. Virtual monitor means virtual space that can be controlled by hand mouse. It is possible that the coordinate on virtual monitor is accurately mapped onto the coordinate on real monitor. Hand-mouse interface based on virtual monitor concept maintains outstanding intuitiveness that is strength of the previous study and enhance accuracy of mouse functions. Further, we increased accuracy of the interface by recognizing user`s unnecessary actions using his concentration indicator from his encephalogram(EEG) data. In order to evaluate intuitiveness and accuracy of the interface, we experimented it for 50 people from 10s to 50s. As the result of intuitiveness experiment, 84% of subjects learned how to use it within 1 minute. Also, as the result of accuracy experiment, accuracy of mouse functions (drag(80.4%), click(80%), double-click(76.7%)) is shown. The intuitiveness and accuracy of the proposed hand-mouse interface is checked through experiment, this is expected to be a good example of the interface for controlling the system by hand in the future.

키넥트를 사용한 NUI 설계 및 구현

이새봄(Sae-Bom Lee),정일홍(Il-Hong Jung) 한국디지털콘텐츠학회 2014 한국디지털콘텐츠학회논문지 Vol.15 No.4

As the use of computer has been popularized these days, an active research is in progress to make much more convenient and natural interface compared to the existing user interfaces such as keyboard or mouse. For this reason, there is an increasing interest toward Microsoft`s motion sensing module called Kinect, which can perform hand motions and speech recognition system in order to realize communication between people. Kinect uses its built-in sensor to recognize the main joint movements and depth of the body. It can also provide a simple speech recognition through the built-microphone. In this paper, the goal is to use Kinect`s depth value data, skeleton tracking and labeling algorithm to recognize information about the extraction and movement of hand, and replace the role of existing peripherals using a virtual mouse, a virtual keyboard, and a speech recognition.

LCD 빔 프로젝터 사용 환경에서 가상 마우스 개발

徐明煥,崔祐榮 明知大學校 産業技術硏究所 2004 産業技術硏究所論文集 Vol.23 No.-

The usage of LCD beam projector is popular because of visualizing presentation data in connection with a computer and its various functions. In case of presentation, however, user's action is restricted by using mouse or remote controller. In this paper, PC-camera is used for more efficient usage of the beam projector and convenient user's action. We developed virtual mouse system composed of an convex mirror indicator and PC-camera. The dectected position of indicator is used for simulating vitual mouse function. Through the simulation, we can get the good result that the system works well with existing presentation software.

한 개의 카메라를 이용한 최소오차 레이저 빔 포인터 위치 검출

이왕헌(Wangheon Lee),이현창(Hyun Chang Lee) 한국컴퓨터정보학회 2007 韓國컴퓨터情報學會論文誌 Vol.12 No.6

강의 및 회의에서 발표자가 파일을 열거나 PC상의 모니터를 직접 조작하기 위하여 진행 중인 발표를 멈추고 PC에 다가가 필요한 조작을 하게 되면 발표의 흐름이 끊기게 된다. 이러한 불편함을 개선하기 위한 여러 방법들이 제안되었으나 주로 레이저 빔 포인터[LBP]에 마우스 기능을 부착하여 처리 하려는 시도들이 대부분이고근본적인 해결은 되지 않고 있다. 본 논문에서는 한 개의 카메라와 영상처리 알고리즘을 적용하여 설치가 간단하고 저가격으로도 실현이 가능하면서도 빔의 검지 위치 정도를 높인 마우스 기능이 부착된 LBP를 제안하고 구현 하였으며, 실험을 통하여 마우스의 위치 인식 오차를 분석하였다. 본 연구의 결과 제안된 LBP가 카메라를 사용 하였음에도 불구하고 조명의 변화나 시야각의 변화에 대해서도 검지된 빔의 위치인식의 반복성과 마우스의 커서로 사용하기에 충분한 고정도의 위치 인식 오차를 보여주고 있음을 확인 하였다. The main stream of presentation is interrupted because of the direct manipulation of their PC frequently so as to control the screen and file open and so on. A variety of products have been developed to solve these inconveniences of the conventional laser beam pointer [LBP] by simply adding a mouse function to the previous LBP. However. the LBPs fully supporting a mouse function are not yet appeared. In this paper. we developed the LBP fully fulfilling a mouse function using mono-camera as well as a robust image processing and analyzed the position detection accuracy. Finally we verified the developed LBP does not only fulfill a mouse function but also solve the defects of the current laser pointer such as inconvenient installation and position detection errors due to the illumination and viewing direction changes.

Virtual Screening and Testing of GSK-3 Inhibitors Using Human SH-SY5Y Cells Expressing Tau Folding Reporter and Mouse Hippocampal Primary Culture under Tau Cytotoxicity

Lin Chih-Hsin,Hsieh Yu-Shao,Sun Ying-Chieh,Huang Wun-Han,Chen Shu-Ling,Weng Zheng-Kui,Lin Te-Hsien,Wu Yih-Ru,Chang Kuo-Hsuan,Huang Hei-Jen,Lee Guan-Chiun,Hsieh-Li Hsiu Mei,Lee-Chen Guey-Jen 한국응용약물학회 2023 Biomolecules & Therapeutics(구 응용약물학회지) Vol.31 No.1

Glycogen synthase kinase-3β (GSK-3β) is an important serine/threonine kinase that implicates in multiple cellular processes and links with the neurodegenerative diseases including Alzheimer’s disease (AD). In this study, structure-based virtual screening was performed to search database for compounds targeting GSK-3β from Enamine’s screening collection. Of the top-ranked compounds, 7 primary hits underwent a luminescent kinase assay and a cell assay using human neuroblastoma SH-SY5Y cells expressing Tau repeat domain (TauRD) with pro-aggregant mutation ΔK280. In the kinase assay for these 7 compounds, residual GSK-3β activities ranged from 36.1% to 90.0% were detected at the IC50 of SB-216763. In the cell assay, only compounds VB-030 and VB-037 reduced Tau aggregation in SH-SY5Y cells expressing ΔK280 TauRD-DsRed folding reporter. In SH-SY5Y cells expressing ΔK280 TauRD, neither VB-030 nor VB-037 increased expression of GSK-3α Ser21 or GSK-3β Ser9. Among extracellular signal-regulated kinase (ERK), AKT serine/threonine kinase 1 (AKT), mitogen-activated protein kinase 14 (P38) and mitogenactivated protein kinase 8 (JNK) which modulate Tau phosphorylation, VB-037 attenuated active phosphorylation of P38 Thr180/ Tyr182, whereas VB-030 had no effect on the phosphorylation status of ERK, AKT, P38 or JNK. However, both VB-030 and VB-037 reduced endogenous Tau phosphorylation at Ser202, Thr231, Ser396 and Ser404 in neuronally differentiated SH-SY5Y expressing ΔK280 TauRD. In addition, VB-030 and VB-037 further improved neuronal survival and/or neurite length and branch in mouse hippocampal primary culture under Tau cytotoxicity. Overall, through inhibiting GSK-3β kinase activity and/or p-P38 (Thr180/Tyr182), both compounds may serve as promising candidates to reduce Tau aggregation/cytotoxicity for AD treatment.

KINECT기반의 3D 인체 제스처마우스 개발

주강, 서래원 배재대학교 공학연구소 2011 공학논문집 Vol.13 No.1

The development of three-dimensional image processing techniques and endless H/W technology advances more and more three-dimensional imaging is increasing the demand for navigation. 3D navigation of the necessary components as a substitute for real world environment, the creation of virtual reality technology allows them to control, as well as control technology must be accompanied by: This is a system that provides a simple 3D background, but in everyday actions that can control the system refers to the instinctive need to develop interface technology. The purpose of this study of human behavior that is closest to the new 3D navigation and control technologies are in development. In this paper, based on the data of KINECT Mouse Gestures such a core technology for the design is about.

SHAKE IT

Seong-Man Cho,Sang-Youn Kim 제어로봇시스템학회 2009 제어로봇시스템학회 국제학술대회 논문집 Vol.2009 No.8

This paper proposes a haptic mouse system and a vibrotatile rendering method for providing haptic sensation when a user shakes a box containing a box containing a tiny object. For the vibrotactile redering, we suggest a hapic model which realistically simulates the haptic behavior of the object of the object in real-time. The proposed haptic model consists of virtual dampers and virtual cylinders in dorder to haptically simulate the haptic behavior of the object having a tiny object. In the proposed system, vibrotactile and thermal sensation are created by eccentric motors and a peltier actuator, respectively. The test for the proposed haptic mouse and the proposed rendering method is accomplished by implementing a "shaking" box application.

영어 초보자의 영어 학습 능력 향상을 위한 딥러닝 기반 가상 튜터 프로그램

주재성(Jae-Seong Ju),홍우진(Woo-Jin Hong),문우혁(Woo-Hyeok Moon),유사라(Sa-Ra Yu),고경수(Kyeong-Soo Ko),고영서(Yeong-Seo Go),장하은(Ha-Eun Jang),김시현(Si-Hyeon Kim),신정훈(Jeong-Hoon Shin),최승호(Seoung-Ho Choi) 한국디지털콘텐츠학회 2023 한국디지털콘텐츠학회논문지 Vol.24 No.5

In learning English, it can be challenging to have conversations with others, even for those who are familiar with the pronunciation. Practicing communication with a real person is not easy for those who are just starting out. To address this issue, we propose a deep learning-based virtual tutor program for English beginners to practice speaking English on their own. The program uses 13 scenarios to simulate conversations, producing direct data. We used deepfake technology to create a virtual human who accurately mimics speech and facial expressions. We also applied a lip mouse model to enhance speech accuracy. Our experimental results showed that the virtual human’s pronunciation closely resembles that of a real person, and the lip mouse model improves speech accuracy compared to when the deepfake model is used alone. Therefore, English beginners can improve their pronunciation and conversation skills in English using our program.

LCD 빔 프로젝터 사용 환경에서 지시자 검출 연구

徐明煥,崔祐榮 明知大學校 産業技術硏究所 2003 産業技術硏究所論文集 Vol.22 No.-

Abstract - Nowadays, the usage rate of LCD beam projector is being increased because of good connection with a computer and various functions. But even though it has many functions, it is almost used only for display. Especially, in case of presentation user's action is limited by using mouse or remote controller. So in this paper pc-camera is used for more efficient usage of the beam projector and convenient user's action. And we have done research an indicator to carry out instruction transmission after fing out position from pc-camera and detect a position which is pointed by it.

KINECT 기반 3D 홀로그래픽과 제스처에 대한 연구

주강(Zhou-Jiang),서래원(Laiwon Seo),노창배(Changbae Roh) 한국디지털콘텐츠학회 2013 한국디지털콘텐츠학회논문지 Vol.14 No.4

Two-dimensional image processing method and tools Rigi then developed a report prepared by a variety of video and three-dimensional images are increasing demands for navigation. The hard part to experience in the real world experience in the virtual environment, and has the purpose to take advantage of. This is a system that provides a simple 3D background, but everyday actions that can control the system with the needs of an instinctive interface technology means. The purpose of this study a variety of human behavior using the Kinect device in action close to the three-dimensional technology to develop a new navigation control is Kinect Holography and 3D images using the input data so that you have the linkage is to design the system.