Scale-aware decomposition of images based on patch-based filtering

진보라 서울대학교 대학원 2015 국내박사

This dissertation presents an image decomposition algorithm based on patch-based filtering, for splitting an image into a structure layer and a texture layer. There are many applications through the decomposition because each layer can be processed respectively and appropriate manipulations are accomplished. Generally, structure layer captures coarse structure with large discontinuities and a texture layer contains fine details or proper patterns. The image decomposition is done by edge-preserving smoothing where structure layer can be obtained by applying smoothing filters to an image and then a texture layer by subtracting the filtered image from the original. The main contribution of this dissertation is to design an efficient and effective edge-preserving filter that can be adapted to various scales of images. The advantage of the proposed decomposition scheme is that it is robust to noise and can be extended to a noisy image decomposition, while conventional image decomposition methods cannot be applied to a noisy image decomposition and conventional image denoising methods are not suitable for image decomposition. To be specific, a patch-based framework is proposed in this dissertation, which is efficient in image denoising and it is designed to smooth an image while preserving details and texture. Specifically, given a pixel, the filtering output is computed as the weighted average of neighboring pixels. For computing the weights, a set of similar patches is found at each pixel by considering patch similarities based on mean squared error (MSE) and other constraints. Then, weights between each patch and its similar patches are computed respectively. With the patch weights, all the pixels in a patch are updated at the same time while adapting to the local pixel weight. For better edge-preserving smoothing, the proposed algorithm utilizes two iterations which are performed through the same smoothing filter with different parameters. Also kernel bandwidth and the number of similar patches are tuned for multi-scale image decomposition. The proposed decomposition can be applied to many applications, such as HDR tone mapping, detail enhancement, image denoising, and image coding, etc. In detail enhancement, the proposed smoothing filter is utilized to extract image detail and enhance it. In HDR tone mapping, a typical framework is used where the smoothing operator is replaced by the proposed one to reduce contrast range of a high dynamic range image to display it on low dynamic range devices. For image denoising, a noisy input is decomposed into structure/texture/noise and the noise layer is discarded while the texture layer is restored through the histogram matching. Also a novel coding scheme named as ``structure scalable image coding scheme'' is proposed where structure layer and salient texture layer are encoded for efficient image coding. Experimental results show that the proposed framework works well for image decomposition and it is robust to the presence of noise. Also it is verified that the proposed work can be utilized in many applications. In addition, by adopting the proposed method in decomposition of a noisy image, both image denoising and image enhancement can be achieved in the proposed framework. Furthermore, the proposed image coding method reduces compression artifact and improve the performance of image coding.

企業이미지(Corporate image)의 影響要因에 관한 硏究

이미라 淑明女子大學校 1993 국내석사

In this paper, we study the constituent parts of the corporate image in the tendency of increasing importance of the corporate image. We show how these factors affect the whole valuation of the corporate image. In order to discuss the these matters, we give five example corporations KEPCO, POSCO, SAMSUNG, DAEWOO and HUYNDAI. We investigate them from the many-sided viewpoints. The epitomes of the results in this research are followed. The 23 items to form the corporate image that abriged 7 costituent parts have a high cofidence on its own costituent parts. The first principal factor in analizing the importance of the corporate image is social confidence factor. The next is, in the order named, corporate reform factor, corporate internal stability factor, national and social contribution, corporate growth factor, external recognition factor, history and investmental motive factor. The objective valuations of the corporate image do not correspond with the subjective factors of the corporate image. The whole image of the domestic corporations are not particularly good. The corporate reform factor mostly influences to valuate the corporate image. The domestic corporations, in comparison with foreign corporations, have merits - diligence of labors, excellence of component people and atmospher of family and defects - lack of research and development, negative attitude of profits reinvestment and instability of the relations between labor and management. In the valuation analisis of the corporate image in population - statistical variable, corporate internal stability factor have a difference by educational level. Corporate reform factor, social confidence factor and corporate growth factor have a difference by income. The rests do not have a intentional difference. With these results, every corporation include above 5 must has a thorough grasp of merits and defects of its own corporations and difference of image thought within the corporations, then they construct and manage the image strategy. At this time. the corporation have to scrutinize all the factors of costituent parts and formed factors of the corporate image, then the corporations have to advance and develop the corporate image.

(A) Study of Image Deblurring with Partial Information

Tayyab Wahab Awan 명지대학교 대학원 2015 국내석사

Recovering a clear image from a degraded image is a challenging problem in computer vision. It is an ill-posed problem so direct methods does not resolve the problem. Most of the existing blind deblurring methods work for single case of blur. The thesis presents a novel method for image blur removal and it works for two different kind of blurred images, motion blur and out-of-focus blur. This method used some prior information from reference image which helps in reconstruction of sharp image. Proposed method used the minimization process to estimate the blur kernel and sharp image. Minimization method needs some regularization terms to reconstruct the blur kernel and sharp image. A new regularization term is introduced in the proposed method and is added to minimization term which helps to reduce the noise and in reconstruction of sharp image. This allows a simple cost formulation to be used for blind deconvolution. After the kernel estimation a non-blind method is used to recover the final deblurred image. Non-blind methods are very much effective to wrong kernel which amplifies the frequency contents which were not attenuated by the real blur kernel and it results of ringing effect in the recovered image. Prior information helps in estimation of a good and less noisy kernel in less iterations. It also helps the non-blind deconvolution process to reduce noise from the recovered image. Experiment shows that the proposed method is more robust and provides better results as compared to previous methods. 질이 좋지 않은 영상으로부터 선명한 영상으로 복구하는 것은 컴퓨터 vision에서 중요한 도전과제이다. 그것은 부적절하게 정립된 문제이기 때문에 직접적인 방법으로 문제를 해결할 수 없다. 대부분의 존재하는 blind deblurring 방법들은 한 개의 흐림 효과의 경우를 위해 동작한다. 논문은 영상 흐림 효과를 제거하는 새로운 방법을 제시한다. 그 방법은 motion blur 와 out-of-focus blur와 같은 두 종류의 blurred 영상을 위해 동작한다. 이 방법은 reference image로부터 사전정보를 사용한다. 그리고 그 영상은 sharp image의 재구성에 도움이 된다. 제시된 방법은 blur kernel 과 sharp image를 추정하기 위해 최소화 과정을 사용했다. 새로운 정규화 term은 제시된 방법에서 소개된다. 그리고 noise를 감소하고 sharp image의 재구성을 도와주는 최소화 term이 추가된다. 이것은 blind deconvolution을 사용하여 단순한 공식화를 가능하게 한다. kernel 추정 후에 non-blind 방법은 최후의 deblurred image로 복원하기 위해 사용된다. Non-blind 방법은 실제 blur kernel에 의해 감소되지 않고 복구된 영상에서 ringing effect가 되는 주파수 내용을 증폭하는 잘못된 kernel에서 매우 효과적이다. 사전정보는 좋고 덜 noisy한 kernel을 적은 반복으로 판정하는 것을 도와준다. 그것은 또한 복구된 영상으로부터 noise를 감소하기 위해 the non-blind deconvolution 과정을 도와준다. 실험은 이전의 방법들과 비교하여 제시된 방법이 더 강력하고 더 좋은 결과를 제공 한다는 것을 보여준다.

Large parallax image stitching and its quantitative assessment

Kyunghwa Jung DGIST 2021 국내박사

본 논문은 시차 이미지 스티칭 방법과 스티칭 방법의 정량적 성능 평가 방법에 관한 연구이다. 곡선 보간 방식의 모핑 방법을 이용하여 큰 시차를 가지는 입력 이미지로 물체 파노라마를 만드는 방법을 제안하였고, 입력 이미지의 시차 레벨을 분석함으로써 시차에 강인한 파노라마 알고리즘의 성능을 정량적으로 평가하는 방법을 제안한다. 파노라마는 크게 배경 파노라마와 물체 파노라마로 나눌 수 있다. 배경 파노라마는 동일한 시점에서 촬영된 이미지들이 합쳐진 광각의 이미지이며, 배경 파노라마는 여러 시점에서 연속적으로 촬영된 비디오 프레임들이 합쳐진 다시점의 이미지이다. 지금까지 물체 파노라마를 만들기 위해서 비디오 프레임과 같이 작은 시차를 가지는 이미지들을 이어 붙여서 고스트 에러를 최소화하였다. 하지만, 내시경 수술과 같이 카메라 위치 변경에 제한이 있을 때는 비디오 기반의 물체 파노라마 생성 방법을 사용할 수 없다. 따라서 우리는 구 형태의 어깨∙무릎 관절경 수술에 사용할 수 있는 이미지 기반의 물체 파노라마 생성 방법을 개발하였다. 제안 방법은 서로 다른 시점에서 촬영한 두 장의 영상을 곡선 보간 방식의 모핑 방법으로 이어 붙임으로써 3 차원 구형 물체에 대한 함몰 왜곡이 적은 물체 파노라마를 생성하는 것이다. 우리는 60°각도 차이를 가진 입력 이미지로 물체 파노라마를 만드는 성능을 기존의 여러 파노라마 알고리즘들의 성능과 비교하였다. 추가적으로 시차 이미지 스티칭 방법의 성능을 객관적·정량적으로 평가하는 방법을 개발하였다. 많은 이미지 스티칭 방법들이 개발되어 왔지만 이들의 성능은 주로 주관적이고 질적으로 평가되어왔다. 스티칭 이미지의 품질을 정량화하기 위해 여러 객관적 평가 방법들이 제안되었지만 입력 이미지의 시차 수준을 고려하지 않고 출력되는 스티칭 이미지 만 분석되어왔다. 따라서, 우리는 입력 이미지의 시차 레벨을 그룹으로 나누고 각 레벨에 대한 스티칭 방법의 성능을 정량적으로 평가하였다. 시차 레벨을 추정하기 위해서 기존의 일치 오류 매트릭과 패치 유사 매트릭을 조합한 매트릭을 사용하였다. 메트릭으로 추정된 입력 이미지의 시차는 크기 및 분산을 기반으로 그룹화되었다. 본 연구에서 총 73 쌍의 입력 이미지의 시차 레벨이 다섯 그룹으로 나뉘어 졌고, 각 시차 그룹에 대한 여러 이미지 스티칭 알고리즘의 성능을 잔류 오정렬 값을 이용해 정량적으로 비교하였다. 우리는 위 평가 방법을 통해 물체 파노라마를 만드는데 사용된 입력 이미지의 시차 레벨이 그룹 G3 에 속하며 이로부터 약 40 %의 시차가 개선되었다는 것을 정량적으로 확인하였다. 제안 파노라마 방법은 제한된 카메라 움직임 환경에서도 다시점의 물체 파노라마를 만들 수 있게 하였고, 높은 시차 개선률로 가장 자연스러운 파노라마를 만들었다. 관절경의 추가 개발을 통해 다시점 파노라마 이미지를 의사들에게 제공한다면 직관적 관절경 수술이 가능할 것이다. 또한, 제안 파노라마 평가 방법은 입력 이미지의 시차 분석을 통해 파노라마 방법의 시차 개선 능력을 정량적으로 평가할 수 있게 하였다. 이는 나아가 이미지 스티칭을 사용하고자 하는 연구자가 주어진 이미지의 시차 수준에 따라 적절한 스티칭 방법을 선택하는데 있어서도 도움을 줄 수 있을 것이다. A parallax image stitching method and its quantitative evaluation method were proposed. The large parallax images were stitched by using an image morphing of a curved interpolation method to create an object panorama, and a performance of the parallax-tolerant image stitching method was quantitatively assessed by analyzing the parallax level of an input image. Traditionally, in order to construct a panoramic image including multiple faces of an object, consecutive video frames must be captured around the object so that images with small parallax can be stitched together to avoid ghost artifacts. However, when a camera has limitation on changing its poses, such as endoscopic surgery, a video-based object panorama generation method cannot be used. Therefore, we developed a method to create an object panorama that can be used in arthroscopic surgery for the spherical joints such as shoulders and knees. The proposed method can create an object panorama for a 3D spherical object by stitching two images captured from different viewpoints using a morphing method of a curved interpolation method. The object panorama produces an unwrapped surface image of a three-dimensional spherical object. Two input images have a larger parallax than the video frames. Therefore, in order to align the overlapping regions of the large-parallax images, an image-morphing method with a curved interpolation line is proposed. The interpolation curve is designed for a spherical target object and it reduces dent distortion. The experimental results for large-parallax images with an angle difference of 60° demonstrate the effectiveness of the proposed method. In addition, we propose a method to quantitatively evaluate the performance of a parallax-tolerant panorama algorithm by analyzing the parallax level of the input image. While parallax-tolerant image stitching is a relatively mature field, the performance of image stitching methods has been assessed subjectively and qualitatively so far. These methods primarily provide the stitched image itself rather than quantitative data to demonstrate the performance. Although several objective assessment methods have been proposed to quantify the quality of stitched images, only the output stitched images have been analyzed without considering the parallax level in each input image. Therefore, we quantify the parallax level of the input images and accordingly cluster the images; this facilitates a quantitative assessment of the various stitching methods for each parallax level. The parallax levels of the images are grouped based on the magnitude and variation in the planar parallax, as estimated with the proposed metric using matching errors and patch similarity. The existing image stitching methods are compared experimentally in terms of the residual misalignment errors, based on 73 pairs of different levels of parallax images originally classified in this study. With the proposed assessment method, we confirmed that the parallax level of the input image used for the object panorama belongs to the group G3 and the parallax is improved by about 40 %. The proposed panorama method could create a multi-view object panorama even in a limited condition of camera movement and make the most natural panorama with a high parallax improvement. If the proposed object panoramic image is provided to surgeons through the further development of arthroscopic system, an intuitive arthroscopic surgery will be possible. In addition, the proposed evaluation method can aid in specifying the parallax tolerant performance and finding an appropriate method depending on the parallax level of given images.

Image stabilization and detail-preserving contrast enhancement for high-performance camera system

김진형 Graduate School, Korea University 2011 국내박사

The goal of this research is to develop algorithms which generate high quality image for high-performance camera system. The proposed methods remove the unintentional camera motion in the input image, and enhance the contrast while preserving the detailed information of the original image. First, this dissertation introduces a highly precise DIS scheme for a hybrid stabilizing system. The stabilizing system adopts a hybrid method of using both OIS and DIS. In the stabilizing system, OIS prestabilizes the original unstable image using gyro-sensors and the resultant image obtained from OIS is post-stabilized using DIS to remove the residual jitters less than one pixel. The proposed DIS, which is newly designed using CGI, can remove not only translational jitters but also rotational ones simultaneously. Second, a novel contrast enhancement scheme using DPS is presented in this dissertation. In order to enhance the contrast, the proposed method adopts contrast stretching since the stretching has a low computational complexity and preserves the global contrast of the original. However, the stretching often produces unpleasant images that do not contain clear details by limiting the output dynamic range. To solve this problem, we propose a new contrast stretching method based on GDP that can enhance the local image contrast while preserving the global contrast as well as the image details. The proposed scheme is useful for enhancement of the image which has low contrast such as IR imagery. Experimental results show that the proposed image enhancement schemes can remove the unintentional camera movement, achieve considerable performance improvement against conventional image enhancement techniques, and outperforms other methods with respect to the visual quality and computational complexity.

人體의 Image를 통한 原始的 感性表現 : 本人의 作品을 中心으로

이명숙 慶星大學校 大學院 1994 국내석사

With my works, I put the stress on the most primitive sensibility of man (imspiration) by using soil, which can give primitive homesicknes that is mild, warm, smooth, strong to the modern people who lost humanity. In the aspect of forming, I took a force at the fertile form that can be ascribed to only humanbeings, and did my best to express the image of breast, stomach, and hip. The image of a woman was formed by the transformed, simplified and curved line by rating out the details of the basic and realistic human body. And then the theme is felt with forming the image of life which is established by a source of life, 'the Mother God of Soil', and the instinctive and generative power of man. These are resulted from the primitive nature of the human body which attains to man's most basic inspiration. Therefore, the motive such as a fertile form of the human body enables men to look into their own insides faded away from the society, and that motive is the restoration of humanity. I analysed 22 pieces of my works, from which I concluded as follows. First, the humen body which has been an obective of formative arts is expressed by the image of humanbeings to look subjective and fertile. But now the human body is far from the objective of reappearance. Second, the transformation process of the human body, such as distorting, omitting and exaggerating the image of feminity, causes the instinctive and generative power with man to make a fertile form. In this way the work influenced on a stone image of a woman, the 'Fertility Symbolism' of the primitive art, enables people to make homony with his life power and the form. In the process of making a work, the principal material is soil which is related to the combination of the basic image and the image composed of a main source of life like breast, stomach and hip. The features of soil express most naturally the round form, the property of a woman. Consequently we can obtain life from the beginning of the world which is contained in soil.

Image Denoising Networks based on the Attention Mechanism for the Enhancement of Natural and Medical Images

Haeyun Lee DGIST 2022 국내박사

영상내 노이즈 제거는 이미지 처리 및 컴퓨터 비전 분야에서 기본적인 작업이다. 노이즈 제거의 중요성 때문에 지난 수십년간 단순한 필터링에서 정교한 학습 기반 접근 방식에 이르기까지 많은 노이즈 제거 방법들이 제안되었다. 딥러닝의 발전과 함께 많은 딥러닝 기반 이미지 노이즈 제거 방법이 제안되고 있다. 최근 몇 년 동안 어텐션 메커니즘 기반 네트워크는 이미지 복원 분야에서 높은 성능을 보여주고 있다. 그러나 이러한 어텐션 기반 영상 복원 방법에 사용되는 어텐션 모듈은 영상 인식 문제를 고려하여 영상 복원에 적합하지 않은 경우가 있다. 따라서 본 논문에서는 영상 노이즈 제거에 적합한 새로운 어텐션 메커니즘을 제안하면서 동시에 제안한 어텐션 메커니즘을 활용한 일반 및 영상 향상을 위한 영상 노이즈 제거 딥러닝 알고리즘을 제안한다. 이를 위해 채널, 셀프 어텐션 모듈과 같은 기존 주의 모듈을 분석했다. 먼저 영상 노이즈 제거에서 채널어텐션 모듈 적용한 뒤 컨텐츠 및 노이즈 정도에 따라 다르게 동작한다는 것을 입증했다. 이 분석을 기반으로 우리는 로컬 적응형 채널 어텐션 모듈을 제안했다. 이 모듈을 기반으로 한 영상 복원 네트워크는 영상 노이즈 제거에서 최신 방법들에 비해 높은 성능을 냈고, 기존의 채널 어텐션 모듈에 비해 1.13배 빠르다는 것을 보였다. 그 다음으로 기존의 셀프 어텐션 모듈은 계산량이 매우 많아 실제로 이용하기에 부적합하다. 그래서 우리는 기존의 셀프 어텐션 모듈의 계산 방법을 개량하여 계산량을 대폭 감소시켰으며, 다중 스케일에 대해 영상의 자기 유사도 특성을 이용하기 위해 다중 스케일 셀프 어텐션 모듈을 새롭게 제안했다. 다중 스케일 셀프 어텐션 모듈 기반의 네트워크는 형광 현미경 영상 노이즈 제거에서 가장 우수한 성능을 보였으며 우리가 제안한 3D 컨포칼 영상에서도 가장 좋은 성능을 보였다. 마지막으로 컨텐츠 인식 어텐션 모듈을 제안하여 초음파 영상에서의 스펙클 노이즈 제거 위한 DCAIP 방법을 제안했다. 콘텐츠 인식 어텐션 모듈은 콘텐츠 종속성을 결정하여 콘텐츠 적응 가중치를 사용했습니다. 제안한 방법은 초음파 이미지의 스페클 감소에서 좋은 성능을 보였고 유방암 분할에서는 AUPRC기준으로 15.89% 성능을 개선했습니다. Image denoising is a fundamental task in the field of image processing and computer vision. Many denoising approaches have been proposed in the past several decades, from simple filtering to sophisticated learning-based approaches. With the development of deep learning, many deep learning-based image denoising methods have been proposed. In recent years, attention mechanism-based networks have been shown to exhibit high performance in the field of image restoration. However, there are cases where the attention module used in these attention-based image restoration methods is not suitable for image restoration by considering the image recognition problem. Thus, there is a need to propose new attention modules suitable for image denoising. To this end, I first analyzed existing attention modules such as the channel and self-attention modules. I proved that the channel attention module has the characteristics of content adaptivity and noise adaptivity. Based on this analysis, I proposed a locally adaptive channel attention module. The proposed attention module improved the performance of the existing channel attention module and enhanced the computation speed of image denoising. Second, I improved the existing self-attention module, which could exploit the self-similarity property of an input image. The existing self-attention module has a disadvantage in that it can only be applied to small-sized images due to its considerable computation cost. Thus, I changed the calculation method to drastically reduce the number of computations and proposed a novel multi-scale self-attention module to exploit self-similarity at different scales. A multi-scale self-attention network based on these modules showed the highest performance for denoising a fluorescence microscopy dataset and a proposed 3D confocal dataset. Finally, I proposed a deep content-aware image prior with a content-aware attention module for despeckling an ultrasound image without a clean image. The content-aware attention module used content-adaptive weights by determining the content dependency. The deep content-aware image prior showed good performance in the speckle reduction of ultrasound images and improved the performance in breast cancer segmentation.

父親이미지에 대한 세대간 요인구조 분석

김명희 東亞大學校 大學院 1996 국내석사

Nowadays it is said that the role of father in a family has changed or has been changing. The purpose of this study is to find serveral factors which are related to Father Image and make the change clear by comparing the present father with the past father. Futhurmore, it Is also pursued which variables affect those factors. The subject of the analysis are 400 fathers and 440 secondary school students. The questionaire consisted of 78 items is used for the survey. Qustions are as follows ; 1. What kinds of factor about father image is appeared? 2. Is there any difference between present father image and past father image? 3. What variables are like which affect Father Image? The brief results of the analysis are summarized like below ; 1. Major factors about father image can be descirbed in terms of these three perspectives. 1) The image of present father toward students : leadership and fidelity, warmth, brightness, stubborness, nonaffinity. 2) The image of present father : leadership and fidelity, warmth, consideration, stubborness, brightness, nonaffinity, etc. 3) The image of past father : leadership and fidelity, warmth, nonaffinity, brightness, intelligence. 2. There are some differences and commonnesses between present father image and past father image. In the present father image, 「warmth」 factor has greater eigen value and factor loading with more extended items, while 「leadership and fidelity」 factor has greater eigen value and factor loading in the past father image. But, the rest of factors have similiar factor loading, eigen value and items in both cases. 3. The image of present father toward students was shown differently according to the school level of students and sex. And the image of present father was shown differently according to the education level and age which was the same variables in case of the image of past father. In conclusion, this study showed that Father Image could be summerized in terms of five to eight factors and the point of view about Father Image has changed and is changing.

Smart Random Erasing for Image Captioning

김연우 서울대학교 대학원 2021 국내석사

Image captioning is a task in machine learning that aims to automatically generate a natural language description of a given image. It is considered a crucial task because of its broad applications and the fact that it is a bridge between computer vision and natural language processing. However, image-caption paired dataset is restricted in both quantity and diversity, which is essential when training a supervised model. Various approaches have been made including semi-supervised and unsupervised learning, but the result is still far from that of supervised approach. While data augmentation can be the solution for data deficiency in the field, existing data augmentation techniques are often designed for image classification tasks and are not suitable for image captioning tasks. Thus, in this paper, we introduce a new data augmentation technique designed for image captioning. The proposed Smart Random Erasing (SRE) is inspired from the Random Erasing augmentation technique, and it complements the drawbacks of Random Erasing to achieve the best performance boost when applied to image captioning. We also derive idea from AutoAugment to automatically search optimal hyperparameters via reinforcement learning. This study shows better results than the traditional augmentation techniques and the state-of-the-art augmentation technique RandAugment when applied to image captioning tasks. 이미지 캡셔닝이란 입력이 이미지로 주어졌을 때, 이미지에 대한 자연어 묘사를 생성하는 머신러닝의 한 과제이다. 이미지 캡셔닝은 시각장애인을 위한 보조자막 생성, 캡션 생성을 통한 검색엔진 성능 향상 등 방대한 어플리케이션을 가질 뿐만 아니라 자연어 처리와 컴퓨터 비전 분야를 연결하는 과제로서 중요성을 지니고 있다. 하지만, 이미지 캡셔닝 모델을 학습하는데 필요한 이미지-캡션의 쌍으로된 데이터셋은 매우 한정되어 있고, 현존하는 데이터셋들 또한 생성되는 문장들의 다양성이 부족하며 이미지 분야도 매우 제한적이다. 이를 해결하기 위해 최근엔 비지도 학습 모델의 연구도 진행되었으나, 현재로서는 지도 학습 모델의 성능을 따라가기엔 아직 한참 부족하다. 데이터 부족 문제를 완화하기 위한 또 다른 방법으로는 데이터 증강 기법이 있다. 최근 이미지 데이터 증강 기법은 AutoAugment, RandAugment 등 활발하게 연구가 진행되고 있으나, 대부분의 연구들이 이미지 분류 문제를 위한 기법들이고, 이를 그대로 이미지 캡셔닝 문제에 적용하기엔 어려움이 있다. 따라서 본 연구에서는 실험을 통해 기존의 데이터 증강 기법이 문제, 모델, 데이터셋에 따라 성능이 매우 달라진다는 것을 확인한다. 그리고 기존의 데이터 증강 기법을 발전시켜 이미지 캡셔닝 문제에 적합한 새로운 기법을 개발하고, 해당 기법의 성능을 실험적으로 검증한다.

Design of a real-time image enhancement processor using Image fusion and full-scale decomposition

윤세환 Graduate School, Korea University 2011 국내박사

In an image acquisition system of a digital camera, image enhancement processes are necessary for image quality improvement since an image signal processor fundamentally focuses on compensating for defects of the image sensor. Image enhancement covers a broad range of techniques such as contrast enhancement, color enhancement, dynamic range extension, and so on. Among these techniques, contrast enhancement is the most critical technique for improving image quality, because the human visual system (HVS) is more sensitive to luminance than to other components such as color information. For this reason, various image enhancement approaches for contrast enhancement have been introduced. In this paper, we focus on three image enhancement approaches, transformation-based, pyramid-based, and Retinex-based. Transformation-based image enhancement such as histogram equalization simply performs global contrast stretching, but may produce monotonous or unstable outputs according to input images. Pyramid-based image enhancement, a multi-scale method, provides outstanding results in terms of both global and local contrast, but causes serious halo effects and costs heavy computation that may lower the possibility of hardware implementation. Retinex-based image enhancement also suffers from a high computational cost due to iterations and multi-scale processing although it possesses the ability to mimic the HVS. The proposed image enhancement algorithm employs a full-scale decomposition framework which is a modified version of a Laplacian-pyramid. In this framework, a low-frequency component enhancement increases contrast and perceptual dynamic range of an input image by combining three transformation images generated from a low-pass image of the input image while a high-frequency component enhancement improves local image information by dealing with Laplacian images of the input image. Thus, the proposed image enhancement algorithm not only overcomes problems such as monotonous or unstable outputs that existing transformation-based image enhancement methods are likely to suffer but also supplements improved local details that those methods cannot provide due to their inherent limitations. Furthermore, this algorithm reconstructs visually pleasing color images using color restoration with appropriate parameter settings. With the proposed algorithm, we design a real-time image enhancement processor to which algorithm optimization is applied in an effort to reduce the hardware overhead. This processor is developed using Verilog hardware description language and implemented on an Altera field-programmable gate array platform. According to software simulation results, the proposed algorithm provides natural and robust image quality and possesses the suitability for video sequences, achieving generally higher performance when compared to existing algorithms. According to hardware implementation results, the proposed processor utilizes 30% to 70% logic elements and consumes 3.5 to 7 times less memory than other processors do in comparison. Therefore, the proposed real-time image enhancement processor is cost-effective and expected to be employed in digital camera applications such as closed circuit television cameras, internet protocol cameras, home video phones, mobile phone cameras, or car black box cameras.