Super-resolution in Music Score Images by Instance Normalization

다국어 초록 (Multilingual Abstract)

The performance of an OMR (Optical Music Recognition) system is usually determined by the characterizing features of the input music score images. Low resolution is one of the main factors leading to degraded image quality. In this paper, we handle the low-resolution problem using the super-resolution technique. We propose the use of a deep neural network with instance normalization to improve the quality of music score images. We apply instance normalization which has proven to be beneficial in single image enhancement. It works better than batch normalization, which shows the effectiveness of shifting the mean and variance of deep features at the instance level. The proposed method provides an end-to-end mapping technique between the high and low-resolution images respectively. New images are then created, in which the resolution is four times higher than the resolution of the original images. Our model has been evaluated with the dataset “DeepScores” and shows that it outperforms other existing methods.

참고문헌 (Reference)

1 Van Khien Pham, "Text Detection based on Edge Enhanced Contrast Extremal Region and Tensor Voting in Natural Scene Images" (사)한국스마트미디어학회 6 (6): 32-40, 2017

2 Nasrollahi, "Super-resolution:A comprehensive survey" 25 : 1423-1468, 2014

3 Y.-W. Tai, "Super-resolution using Edge Prior and Single Image Detail Synthesis" 2400-2407, 2010

4 Borman, "Super-Resolution from Image Sequences - A Review" 374-378, 1998

5 Zheng Xu, "Strong Baseline for Single Image Dehazing with Deep Features and Instance Normalization" 2018

6 Yang, "Single-image super-resolution: A benchmark" 4 : 372-386, 2014

7 QuangNhat Vo, "Recognition of Music Scores with Non-Linear Distortions in Mobile Devices" 77 : 15951-15969, 2018

8 Wenzhe Shi, "Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network" 2016

9 Christian Ledig, "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network" 105-114, 2017

10 K. Zhang, "Multi-scale dictionary for single image super-resolution" 1114-1121, 2012

11 Son Tung Trieu, "Machine Printed and Handwritten Text Discrimination in Korean Document Images" (사)한국스마트미디어학회 5 (5): 30-34, 2016

12 S. Baker, "Limits on super-resolution and how to break them" 24 : 1167-1183, 2002

13 K. Gregor, "Learning fast approximations of sparse coding" 399-406, 2010

14 C. Dong, "Learning a deep convolutional network for image super-resolution" 8692 : 184-199, 2014

15 H. Yue, "Landmark image super-resolution by retrieving web images" 22 : 4865-4878, 2013

16 Dmitry Ulyanov, "Instance Normalization: The Missing Ingredient for Fast Stylization"

17 J. Sun, "Image super-resolution using gradient profile prior" 2008

18 C. Dong, "Image super-resolution using deep convolutional networks" 38 : 295-307, 2016

19 Zhou Wang, "Image Quality Assessment: From Error Visibility To Structural Similarity" 13 : 600-612, 2004

20 Yuxin Wu, "Group Normalization" 1-14, 2018

21 C. E. Duchon, "Filtering in One and Two Dimensions" 18 : 1016-1022, 1979

22 Farsiu, "Fast and robust multiframe super-resolution" 13 : 1327-1344, 2004

23 Ankit Lat, "Enhancing OCR Accuracy With Super Resolution" 3162-3167, 2018

24 Lim, "Enhanced Deep Residual Networks for Single Image Super-Resolution" 1132-1140, 2017

25 Jorge Calvo-Zaragoza, "End-to-End Optical Music Recognition Using Neural Networks" 472-477, 2017

26 J. Kim, "Deeply-recursive convolutional network for image super-resolution" 1637-1645, 2016

27 Lukas Tuggener, "DeepScores - a dataset for segmentation, detection and classification of tiny objects" 3704-3709, 2018

28 Sergey Ioffe, "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift" 37 : 448-456, 2015

29 D. Kingma, "Adam: A method for stochastic optimization" 2014

30 J. Kim, "Accurate image superresolution using very deep convolutional networks" 1646-1654, 2016

31 Luu Ngoc Do, "A Covariance-matching-based Model for Musical Symbol Recognition" (사)한국스마트미디어학회 7 (7): 23-33, 2018