(An) integrated deep learning framework for robust and real-time taillight detection

Jeon, Hyungjoon Sungkyunkwan University 2023 국내박사

In this paper, we present a deep learning model for high-accuracy, high-speed detection of vehicle taillights in traffic. The model consists of three major modules: the lane detector, the car detector, and the taillight detector. Unlike most previously proposed algorithms where hand-coded schemes are used, we have adopted a data-driven approach. Both the pipelined approach and data-driven approach are necessary since we need to deal with both false positives and detection misses without complex hand-crafted logic. Two different implementations are introduced in this paper. In the first implementation, while lane detection was performed using hand-crafted algorithm, we used the ResNet-RRC as the deep neural network for car and taillight detection. In the second implementation, the PINet was used for lane detection, and the YOLOv7 was used for car and taillight detection. The robustness of our model was verified using datasets from Sungkyunkwan University (SKKU) as well as the Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI). Our model works well even in hostile conditions. It achieves detection rates as high as 100% in testing with the SKKU dataset. When using the KITTI 2D Object dataset, the model achieves a taillight detection rate of 96%. The model achieves 100% taillight detection rate on a certain, small subset of the KITTI Tracking dataset. The system achieves real-time speeds not only on large-scale computers but also on embedded machines such as NVIDIA® Jetson™ Orin. 본 논문에서는 교통 상황에서 차량 미등을 고정밀, 고속으로 감지하기 위한 딥러닝 모델을 제시한다. 이 모델은 차선 인식기(lane detector), 차량 인식기(car detector), 그리고 미등 인식기(taillight detector)의 세 가지 주요 모듈(module)로 구성된다. 손으로 코딩한 체계가 사용되는 이전에 제안된 대부분의 알고리즘과 달리, 오인식 및 인식 실패 사례에 대해 복잡한 손 코딩 논리 없이도 예방할 필요가 있기 때문에, 세 모듈에 걸친 파이프라인 접근 방식 및 데이터 기반 접근 방식을 채택하였다. 본 논문에서는 두 가지 방식으로 구현된 모델이 제안된다. 첫 번째 구현에서는 손으로 코딩된 차선 인식기를 사용한 다음 RRC(Recurrent Rolling Convolution) 아키텍처와 차량 경계 감지를 위한 추적 메커니즘을 사용하고, 그 후 동일한 RRC 아키텍처를 사용하여 감지된 차량의 미등 영역을 추출한다. 두 번째 구현에서는 차선 인식기로 PINet(Point Instance Network)를 사용하고, 차량 및 미등 인식기로 YOLOv7을 사용한다. 이 모델의 견고성은 우리 연구팀에서 직접 촬영한 데이터셋(dataset)과 칼스루에 공과대학 및 도요타 공과대학 (KITTI)에서 수집한 데이터셋으로 검증되었고, 적대적인 조건에서도 잘 작동한다. 우리 데이터셋을 사용한 실험에서 최대 100%의 인식률을 달성하였고, KITTI 2D Object 데이터셋을 사용한 실험에서 96%의 인식률을 달성하였다. 해당 모델은 KITTI Tracking 데이터셋의 서브셋(subset)에서 100% 미등 인식률을 달성한다.

Effects of GRP78 expression on cell proliferation in chicken embryo fibroblast cells

Jeon, Mihyang Sungkyunkwan University 2014 국내석사

Pedestrian detection based on deep learning

Jeon, Hyungmin Sungkyunkwan university 2019 국내석사

While it is a hot issue whether cars could drive by themselves in emergent situations without any kind of human interference, pedestrian detection is the key technology in autonomous driving cars. Though current pedestrian detection technologies have come to a point in which they are accurate in normal conditions and surroundings, existent systems are inaccurate in harsh situations, such as when there are too many pedestrians, when there is too much light or when it is too dark, or when it is raining or snowing heavily. This problem may be solved by integrating deep learning and combining a new type of local pattern with the RGB raw image as input, instead of using just the RGB image as input. We will introduce a new type of local pattern called Triangular Patterns, which is effective in extracting more detailed and stable features from local regions. Here in this paper, we propose a pedestrian detection system in which deep learning is used, along with combining the RGB raw image with Triangular Patterns for input.

Design and implementation of denoising hardware architecture

Jeon, Byungmoo Sungkyunkwan University 2013 국내석사

영상처리에서 노이즈 제거는 물체 추적, 스테레오 비전 그리고 의료 영상 복원 등 다양한 분야에서 요구된다. 영상처리에서의 정확한 결과를 획득하기 위해서는 다양한 전처리 과정이 필요하다. 우리는 영상 내 노이즈 제거를 위한 Total Variation의 처리 속도를 개선한 전용 하드웨어 구조를 FPGA 상에 설계하였다. 제안된 시스템에서는 VGA급(640ⅹ480) 해상도 영상을 처리할 수 있도록 설계되었다. 영상 내 노이즈 제거는 노이즈가 있는 영상을 입력받아 동일한 연산을 10회 후에 출력한다. 영상 내 노이즈 제거의 첫 번째 과정에서는 중심픽셀에서 좌측 및 아래측 픽셀과의 픽셀 값 차이를 획득한다. 두 번째 과정에서는 중심픽셀에 픽셀 차이에 파라미터를 곱하여 그 값을 중심픽셀에 더한다. 중심픽셀에 더해지는 값은 픽셀 값 차이와 곱해지는 파라미터에 따라서 다르게 반영된다. 이 두 과정을 10회 반복하면서 영상 내 노이즈를 제거한다. 노이즈 제거는 복잡한 연산과 높은 연산 성능을 필요로 하지만 전용하드웨어 설계를 통하여 노이즈 제거 성능을 개선하였다. 높은 연산 능력을 요구하는 노이즈 제거의 실시간 처리 성능을 획득하기 위하여 전용 하드웨어를 설계하였다. 제안된 알고리즘의 많은 클럭의 연산 지연을 최소화 하기 위해서 파이프라인구조로 하드웨어를 설계하였다. 제안된 시스템은 VGA급(640ⅹ480) 해상도 영상을 250Mhz의 속도로 동작할 수 있다. Noise removal in image processing is required in a variety of fields such as object tracking, stereo vision and medical image reconstruction. To obtain accurate results, various video pre-processing is required. We propose a hardware architecture using FPGA to improve the processing speed with the Total Variation algorithm for noise removal from images. The proposed system can process 640ⅹ480 resolution images. After 10 cycles of operations, noise is removed from the noisy input image. First, the right, bottom and center pixel values and their differences are obtained. Second, pixels are added to the center of the operation parameters, and the difference between them and the values of the surrounding pixels. The operation parameters and the difference of the values of the surrounding pixels are reflected in the following operations. This process is repeated 10 times to remove noise in a noisy input image. The noise removal performance of this system is better than the performances with other systems. This system needs to carry out complex processes that require considerable computing power. We implemented the proposed system in hardware for real-time processing. The processing delay was 0.8ms. We designed a pipelined architecture to delay the operation. The proposed system can process images with resolution of 640 ⅹ480 at 250Mhz.

(A) deep learning framework for robust and real-time taillight detection under various road conditions

Jeon, Hyungjoon Sungkyunkwan university 2019 국내석사

The objective of this paper is to present a model with three major modules—lane detector, car detector, and taillight detector—that ensures both high accuracy and high speed in detecting vehicle taillights in traffic. Taillight detection is important for automated driving. Many algorithms have been proposed for taillight detection, but code-driven scheme was almost always used in these algorithms. To enable more robust detection of taillights, we switched this code-driven scheme to data-driven approach in the present study. Data-driven scheme was implemented in both the car and taillight detection modules. We first used a discretely designed lane detection module, adopted the Recurrent Rolling Convolution (RRC) architecture and tracking mechanism for detecting car boundaries, and then used the same RRC to extract taillight regions and their illumination states upon the detection of cars. For experiments, a dataset obtained by our lab was used for training the RRC network. The robustness of our model was verified by testing on both our dataset and the dataset from the Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI). Results from our model show that lane detection and car detection with tracking can improve both speed and accuracy of taillight detection. In addition, the results reveal that our model works well under hostile conditions, having accuracies as high as 94% when using the dataset from our research team at Sungkyunkwan University (SKKU). Moreover, when using the KITTI dataset, the model has 100% taillight detection accuracy for cars under normal conditions.

Robust physiological signal analysis : design and applications with machine learning

전태균 Gwangju Institute of Science and Technology 2016 국내박사

High nitrogen-induced changes in rhizosphere microbial community structure alter disease susceptibility to the rice blast

Mehwish, Roy 영남대학교 대학원 2024 국내박사

Use of nitrogen fertilizer has been a key agricultural practice in promoting plant growth and crop yield, although excess nitrogen has long been known to have the potential to alter the outcome of plant-pathogen interactions. In rice plants, it has been well established that excess nitrogen application increases the susceptibility of the plants to the rice blast disease, which is caused by a model plant pathogenic fungus, Magnaporthe oryzae. Despite its implication in sustainable crop production, mechanisms that underlie such increase in disease susceptibility remain to be fully elucidated. Here I investigated how nitrogen supply can impact rhizosphere microbiome of rice, and whether such changes in microbiome can lead to increase in susceptibility to the rice blast disease. Using an experimental system where increase in susceptibility to M. oryzae can be monitored without causing apparent physiological aberration, both the bacterial and fungal community structures of rhizosphere was assessed and compared by amplicon sequencing of 16S rRNA and ITS gene, respectively. The analysis showed that rhizosphere community structures are significantly affected by the amount of nitrogen supply and pathogen infection. Analysis of the defense-related genes in rice plant revealed that some of the key genes including a salicylic acid (SA) pathway gene don’t seem to be as inducible by the pathogen infection under high nitrogen condition as it should be. In consistent with such observation, SA levels showed significant reduction in leaves of plants grown under high nitrogen condition, compared to the control plants. Addition of microbial fraction separated from the inoculated plants under high nitrogen condition to the control plants was able to recapitulate increased susceptibility to the rice blast disease. Taken together, my study suggests that alteration of rhizosphere microbiome caused by excessive nitrogen fertilization may be in part responsible for nitrogen-induced susceptibility in the rice plants.

Engineering of tungsten oxide nanoarchitectonics as a dual-function material for electrochromic energy storage applications

Pritam J., Morankar 영남대학교 대학원 2024 국내박사

Characterization of bioactive components from Ishige okamurae and their biological activity

Jiang, Yun Fei 제주대학교 2020 국내박사

Seaweeds are one of the crucial marine living renewable resources as they are able to produce a great variety of secondary metabolites characterized by a broad spectrum of biological activities. Ishige okamurae (I. okamurae) is a member of the family Ishige as an edible brown alga, have attracted attention as various natural compounds with their biological activities. In this study, new separation method was suggested to increase efficiency and improve the isolation performance using Pure C-850 FlashPrep system. In this study, Pure C-850 FlashPrep system was suggested to increase efficiency and improve the isolation of natural compounds from I. okamurae. A novel series of polyphenols, named as Ishophloroglucin; Ishophloroglucin A (IPA), Ishophloroglucin B (IPB), Ishophloroglucin C (IPC), and Ishophloroglucin D (IPD), and another two known compounds which is α/β-adenosine and Diphlorethohydroxycarmalol (DPHC) were isolated and identified. Their structure was elucidated by 1D- and 2D- Nuclear magnetic resonance (NMR), including 1H, 13C, HMBC, and COSY, as well as liquid chromatography-tandem mass spectrometry (LC-MS/MS). As folk medicine, I. okamurae with various biological effects grows abundantly on the shores of Japan, north of China, and Korea where it is consumed as a part of daily diet. Appropriate quality control operations shall be employed to ensure that food having uniformed nutrients is suitable for human consumption. This study was conducted to establish an HPLC analysis method for determination of marker compounds as part of materials standardization for development of health functional food materials from I. okamurae. DPHC was selected as a marker compound due to it is more stable compared to IPA on the thermal stability analysis. The method was validated by system suitability, specificity, linearity, limits of detection (LOD), limits of quantification (LOQ), precision, and accuracy. The method showed high linearity of the calibration curve with a coefficient of correlation (R2) of 0.9999, and LOD and LOQ was 0.144 μg/mL and 0.435 μg/mL, respectively. Relative standard deviation values from precision was less than 0.622 %. Recovery rates of DPHC was 106.35 - 107.82 % within 100 ± 20%. An optimized method for extraction of DPHC in I. okamurae was established through diverse extraction conditions, and the validation indicated that the method is accurate and sensitive for the determination of marker compounds in I. okamurae to develop a health functional food material. Polyphenols, a group of phloroglucinol (1,3,5-trihydroxybenzene), are the dominant polyphenolic secondary metabolites found in marine brown algae. IPA as a dominant polyphenol in I. okamurae, was investigated their anti-melanogenesis effects. Through the comparative molecular docking analysis of IPA to tyrosinase (3NM8), IPA was further investigated their inhibitory effect against tyrosinase activity and melanin formation that causes an increase anti-melanogenesis effect in both murine melanoma cells in-vitro model and zebrafish in-vivo model. The α/β-adenosine (1:1 ratio) was firstly identified from I. okamurae, which is known as a naturally occurring substance that relaxes and dilates blood vessels in angiogenesis. To characterize the vasodilation effect of α/β-adenosine in endothelial dysfunction of angiogenesis induced by Particulate matter (PM) exposure, commercial isomer β-adenosine was also investigated as well in human endothelial cell line EA.hy926 in-vitro model and transgenic zebrafish in-vivo model. In the last study, fucoidan is an interesting group of bioactive sulfated polysaccharides in brown seaweeds. The current study highlights the enrichment and extraction of fucoidan from I. okamurae through enzyme-assistant extraction using Celluclast enzyme. The structural characterization of fucoidan was performed using FTIR and NMR spectroscopy, their monosaccharide compositions was analyzed by HPAE-PAD, the molecular weight distribution was performed by agarose gel electrophoresis and the sulfate content was analyzed as well. The purified fucoidan and its anti-inflammatory activity was investigated in-vitro and in-vivo. In summary, this research suggested various natural compounds from I. okamurae isolated by a flash prepare system and their potential uses as marker compounds or pharmaceutical and nutraceutical agents. Therefore, the purpose of this research is to discuss the potential health benefits of I. okamurae to be used as a functional material in industries such as functional foods, nutraceuticals, and cosmeceuticals to utilize this precious natural resource for future generations.

Development of light-activated adhesive protein-based hydrogel platforms for regenerative wound closure

전은영 Pohang University of Science and Technology 2018 국내박사

In daily life, all wounds require immediate and proper closure to reduce complications (e.g., sepsis and multi-organ failure) and financial burden and to improve healing quality. The primary goal of wound closure is maximal restoration of tissue structure and function, but recently, facilitating healing process and minimizing aesthetic damage become important for advanced wound care. However, due to limitations of existing wound closure options, there are still significant medical demands for sutureless closure of external/internal wounds with both functional and cosmetic satisfaction. Despite excellent mechanical support and low dehiscence rate, sutures and staples are undesirable for where fluids or gasses leak, delicate organs and inaccessible tissues with unwanted side effects such as ischemia, scar formation, and infection by excessive tissue penetration. Medical tissue adhesives including cyanoacrylates and fibrin glue are susceptible to failure of appropriate wound approximation and healing due to initial uncontrollable polymerization and resultant too rigid or brittle surface that cannot accommodate dynamic movement of tissues. As a result of excessive wound repair, abnormal scarring after massive and deep dermal injuries can cause significant physical dysfunction and psychological/aesthetic damage. Unfortunately, due to poor understanding of the underlying molecular basis of wound repair involving in scarring, current clinical scar managements, including invasive (surgery, laser, chemical peeling, injection) and non-invasive methods (silicone gel sheeting, compression therapy, topical drug delivery), are intended to ameliorate the appearance of already established scars. In recent researches with advanced tissue engineering strategies, there are still limitations such as unclear action mechanism, poor integration with host tissue, and side effects from high doses and administration frequency for desired efficacy. Hydrogels have emerged as regenerated templates for wound management and tissue regeneration. Hydrogels mean macromolecular networks stabilized by physical/chemical crosslinkings between hydrophilic polymers with high water content. They possess valuable properties such as hydrophilic nature, similar mechanical characteristics to soft tissue, and efficient transportation of oxygen and encapsulated bioactive molecules. Recently, specific stimuli-responsive hydrogels (e.g., temperature, light, and pH) have been developed for on-demand in situ crosslinking with injectability and smart release of bioactive molecules in various biomedical applications. Particularly, tissue adhesive hydrogel can widely act as an adhesive for wound closure, hemostat, sealant or adhesive medium to bond other restorative materials to tissues for long-term stability. Mussel adhesive proteins (MAPs) secreted from marine mussels have been considered as ideal bioadhesives due to strong adhesion on diverse surfaces, biodegradability, biocompatibility, and water resistance. Previously, to overcome extremely low yield of natural extraction process, recombinant hybrid MAP was successfully mass-produced in a bacterial system for practical applications, which has shown to be biodegradable, biocompatible, and strongly adhesive to various surfaces In this dissertation, inspired from natural organisms and healing mechanisms, we developed light-activated MAP-based hydrogel-forming adhesive platforms using molecular biotechnology and/or surface micro-topography for regenerative wound closure including an instant glue, a scar-preventive glue, and a microneedle (MN) type of adhesive tape with improved wet adhesion and transdermal drug delivery. Firstly, Tyr-rich recombinant hybrid MAP was developed into an instant glue using a visible light-activated Tyr-Tyr crosslinking system, which employs two intriguing strategies from nature; MAP-inherent strong surface adhesion and mechanically stable dityrosine crosslinks found in insects. The light-activated MAP-based bioadhesives (LAMBA) achieved high-order crosslinks within 5 s of blue light irradiation and complete closure of a full-thickness skin incision within 60 s via covalent crosslinking between Tyr residues of MAP and tissues. Unlike tissue adhesives with uncontrollable polymerization, the remarkably fast and on-demand gelation system with sufficient elastic modulus was beneficial to sutureless wound closure under in vivo dynamic environments. Besides, the hydrogel-forming glue could accelerate healing by offering an instructive moist environment for cell behaviors compared to a brittle plastic cover for cyanoacrylate glue and a blood clotting scab for fibrin glue. Therefore, the LAMBA system could be applied to various medical practice that requires immediate and stable tissue adhesion ranging from facial laceration repair in emergency department to fixation of implants. Next, based on LAMBA system, we created a natural healing-inspired scar-preventive glue consisting of a newly designed collagen-binding MAP and dermatan sulfate with the fundamental roles of decorin, a collagen-targeting proteoglycan, in collagen reorganization as inspiration. Collagen reorganization in a remodeling phase is one of important factors responsible for scar formation with abnormal cell-matrix interactions and altered cellular phenotypes. Our collagen-targeting glue could behave similarly to decorin by specifically binding to type I collagen in a dose-dependent manner, regulating the rate and the degree of fibrillogenesis, and providing sufficient biochemical cues for scarless wound healing. In a rat full-thickness excisional skin defect model, the collagen-targeting glue clearly acted to encourage initial wound healing as defined by effective reepithelialization, neovascularization, and rapid collagen synthesis during inflammatory and proliferative phases and minimize scar formation as demonstrated by uniformly packed collagen fibrils, a restoration of healthy dermal architecture and decreased expression of fibrogenic factors during the remodeling phase. Thus, the scar-preventative glue is expected to serve as reliable guidance for further investigations of other clinical problems with heavy scarring, such as diabetic ulcers, myocardial infarction, and corneal scarring. For regenerative closure of more demanding wounds where air/fluids leak or diseases present under wet/dynamic environments (e.g., gastrointestinal perforation and pneumothorax), a microneedle technology was combined to LAMBA system to develop a new adhesive platform with tissue anchorable and/or transdermal drug deliverable MNs. Inspired by swelling proboscis of endoparasitic worms for immobilization to host’s tissue, we designed a double-layered hydrogel-forming MN patch consisting of MAP-based swellable sticky shell and silk fibroin (SF)-based rigid core. Unlike medical tape vulnerable to wet condition, the MN patch achieved superior wet tissue adhesion via synergistic effects from both physical entanglements from selective swelling of MN tips and MAP-derived intrinsic strong surface adhesion. Moreover, in vivo performance clearly suggested great advantages such as simple and fast application, less tissue damage via distribution of mechanical stress along the wound, tight sealing against liquid leaks, and transdermal drug delivery over other available options such as sutures, staples, and glues. Collectively, for regenerative closure of external/internal wounds, we developed bio-inspired light-activated MAP-based hydrogel-forming adhesive platforms including an in situ instant glue, a scar-preventive glue, and a adhesive MN patch. Inspired by insects’ structural crosslinks, the LAMBA could offer on-demand instant wound closure via stable surface adhesive and cohesive bonds without the aid of surgical devices. Furthermore, the collagen-targeting hydrogel glue could act as a surrogate ECM to facilitate wound healing and prevent scarring for excisional skin defects. For more demanding wounds under wet/dynamic environments, swellable adhesive MN patch will be beneficial in that tight wound sealing against liquid leakages via improved surface adhesion from physical interlocking and additional transdermal drug delivery. In conclusion, our different types of light-activated adhesive hydrogel platforms have great potential of practical uses for wound care for burns, waterproof adhesives, sealant for perforations or anastomosis leaks, tissue grafting, and direct local delivery of therapeutics.