Inter-layer Texture and Syntax Prediction for Scalable Video Coding

Lim, Woong,Choi, Hyomin,Nam, Junghak,Sim, Donggyu The Institute of Electronics and Information Engin 2015 IEIE Transactions on Smart Processing & Computing Vol.4 No.6

In this paper, we demonstrate inter-layer prediction tools for scalable video coders. The proposed scalable coder is designed to support not only spatial, quality and temporal scalabilities, but also view scalability. In addition, we propose quad-tree inter-layer prediction tools to improve coding efficiency at enhancement layers. The proposed inter-layer prediction tools generate texture prediction signal with exploiting texture, syntaxes, and residual information from a reference layer. Furthermore, the tools can be used with inter and intra prediction blocks within a large coding unit. The proposed framework guarantees the rate distortion performance for a base layer because it does not have any compulsion such as constraint intra prediction. According to experiments, the framework supports the spatial scalable functionality with about 18.6%, 18.5% and 25.2% overhead bits against to the single layer coding. The proposed inter-layer prediction tool in multi-loop decoding design framework enables to achieve coding gains of 14.0%, 5.1%, and 12.1% in BD-Bitrate at the enhancement layer, compared to a single layer HEVC for all-intra, low-delay, and random access cases, respectively. For the single-loop decoding design, the proposed quad-tree inter-layer prediction can achieve 14.0%, 3.7%, and 9.8% bit saving.

Incorporating inter-relationships between different levels of genomic data into cancer clinical outcome prediction

Kim, D.,Shin, H.,Sohn, K.A.,Verma, A.,Ritchie, M.D.,Kim, J.H. Academic Press 2014 Methods Vol.67 No.3

In order to improve our understanding of cancer and develop multi-layered theoretical models for the underlying mechanism, it is essential to have enhanced understanding of the interactions between multiple levels of genomic data that contribute to tumor formation and progression. Although there exist recent approaches such as a graph-based framework that integrates multi-omics data including copy number alteration, methylation, gene expression, and miRNA data for cancer clinical outcome prediction, most of previous methods treat each genomic data as independent and the possible interplay between them is not explicitly incorporated to the model. However, cancer is dysregulated by multiple levels in the biological system through genomic, epigenomic, transcriptomic, and proteomic level. Thus, genomic features are likely to interact with other genomic features in the different genomic levels. In order to deepen our knowledge, it would be desirable to incorporate such inter-relationship information when integrating multi-omics data for cancer clinical outcome prediction. In this study, we propose a new graph-based framework that integrates not only multi-omics data but inter-relationship between them for better elucidating cancer clinical outcomes. In order to highlight the validity of the proposed framework, serous cystadenocarcinoma data from TCGA was adopted as a pilot task. The proposed model incorporating inter-relationship between different genomic features showed significantly improved performance compared to the model that does not consider inter-relationship when integrating multi-omics data. For the pair between miRNA and gene expression data, the model integrating miRNA, for example, gene expression, and inter-relationship between them with an AUC of 0.8476 (REI) outperformed the model combining miRNA and gene expression data with an AUC of 0.8404. Similar results were also obtained for other pairs between different levels of genomic data. Integration of different levels of data and inter-relationship between them can aid in extracting new biological knowledge by drawing an integrative conclusion from many pieces of information collected from diverse types of genomic data, eventually leading to more effective screening strategies and alternative therapies that may improve outcomes.

H.264/AVC의 실시간 압축을 위한 고속 인터 예측 부호화 기술

김영현,최현준,서영호,김동욱,Kim, Young-Hyun,Choi, Hyun-Jun,Seo, Young-Ho,Kim, Dong-Wook 한국통신학회 2006 韓國通信學會論文誌 Vol.31 No.11C

This paper proposed a fast algorithm to reduce the amount of calculation for inter prediction which takes a great deal of the operational time in H.264/AVC. This algorithm decides a search range according to the direction of predicted motion vector, and then performs an adaptive spiral search for the candidates with JM(Joint Model) FME(Fast Motion Estimation) which employs the rate-distortion optimization(RDO) method. Simultaneously, it decides a threshold cost value for each of the variable block sizes and performs the motion estimation for the variable search ranges with the threshold. These activities reduce the great amount of the complexity in inter prediction encoding. Experimental results by applying the proposed method .to various video sequences showed that the process time was decreased up to 80% comparing to the previous prediction methods. The degradation of video quality was only from 0.05dB to 0.19dB and the compression ratio decreased as small as 0.58% in average. Therefore, we are sure that the proposed method is an efficient method for the fast inter prediction. 본 논문에서는 H.264/AVC에서 가장 많은 연산량을 차지하는 인터 예측(inter prediction)을 고속으로 수행할 수 있는 방법을 제안하였다. 제안한 방법은 율-왜곡 최적화 기법(Rate-Distortion Optimization, RDO)이 적용된 JM(Joint Model)의 FME(Fast Motion Estimation)를 대상으로 예측된 움직임 벡터의 방향성을 고려하여 탐색영역을 결정한 후 적응적인 후보 나선형 탐색을 수행한다. 동시에 가변 블록 크기에 대하여 비용함수의 임계값(threshold)을 결정한 후 가변 구간 움직임 탐색을 수행함으로써 인터 예측의 부호화 복잡도를 감소시킨다. 다양한 영상들을 대상으로 실험한 결과 기존의 예측 방식에 최대 80%의 연산량을 줄일 수 있음을 확인하였다. 이에 따른 화질 열화는 평균 $0.05dB{\sim}0.19dB$에 불과하며, 압축률은 평균 0.58%의 미미한 감소를 보임으로써, 제안한 방법이 고속 인터 예측 알고리즘으로 매우 효율적인 방법임을 확인하였다.

GOP 기반의 적응적 인터 예측을 이용한 다시점 비디오의 효율적인 H.264/AVC 부호화

이정호,조익환,이웅호,정동석,Lee, Jung-Ho,Cho, Ik-Hwan,Lee, Woong-Ho,Jeong, Dong-Seok 한국통신학회 2006 韓國通信學會論文誌 Vol.31 No.12C

본 논문에서는 H.264/AVC의 다시점 비디오 부호화의 고속화 방법을 제안한다. 다시점 비디오는 부호화해야 할 데이터의 양이 일반 비디오보다 더 많으며, 전형적인 비디오 부호화 방법에서 사용하는 인터-움직임 예측에 덧붙여 인터-부등 예측을 이용하기 때문에 예측해야할 정보가 막대하게 커진다. 실험을 통해 비디오 부호화에서 사용되는 각 예측의 효율이 인트라, 인터-부돈, 인터-움직임 그리고 인터-skip 순으로 점점 좋아진다는 점에 착안하여 각 GOP 단위로 적응적인 문턱값을 계산하여 인터-부등 및 인트라 예측을 조기 종료하는 고속화 알고리즘을 제안하였다. 실험 결과 제안하는 방법은 여러 실험 비디오에 대해서 기존의 방법에 비해 약 32%가량 개선된 연산 속도를 보였으며, 비트량 및 왜곡의 증가는 상대적으로 적었다. This paper proposes a fast encoding algorithm of H.264/AVC multi-view video coding. The amount of data to be encoded for a multi-view video is much more than normal video's data, and the amount of information to be predicted is enormous because of the multi-view video coding uses inter-disparity prediction in addition to inter-motion prediction in conventional video coding. We noticed through an experiment that the efficiency of prediction is getting better in order of intra, inter-disparity, inter-motion, and inter-skip, and proposes a early termination algorithm by means of estimate the adaptive threshold within a GOP unit. In the experiments, the proposed algorithm shows improved processing speed about 32% compared to existing method, and increased amount of bits and distortions are relatively disregardable.

H.264/AVC Encoder용 저전력 IP 설계 및 FPGA 구현

장영범(Young Beom Jang),최동규(Dong Kyu Choi),한재웅(Jae Woong Han),김도한(Do Han Kim),김비철(Bee Chul Kim),박진수(Jin Su Park),한규훈(Kyu Hoon Han),허은성(Eun Sung Hur) 대한전자공학회 2008 電子工學會論文誌-SP (Signal processing) Vol.45 No.5

이 논문에서는 제안한 H.264/AVC 인코더의 서브 블록인 Inter prediction 블록, Intra prediction 블록, 디블로킹 필터블록, Transform & Quantization 블록에 대한 저전력 구조를 FPGA로 구현하였다. Inter/Intra prediction 블록에서는 분산연산방식을 통해 가산기의 수를 줄여 60.2%의 면적감소효과를 나타내었으며, 디블로킹 필터블록에서는 하드웨어 공유를 위한 MUX를 사용하여 덧셈연산의 수를 44.3%감소시켰다. 또한, Transform & Quantization 블록에 사용되는 곱셈연산을 CSD와 CSS방식으로 수행하여 면적을 크게 차지하는 곱셈기를 사용하지 않았다. 제안된 저전력 IP들을 사용하여 FPGA(Field Programmable Gate Array)와 ARM 프로세서 기반의 H.264/AVC 인코더를 구현하였다. Baseline Profile을 사용하였고 FPGA와 ARM프로세서가 연동하는 Platform으로 구현하였다. Platform을 사용한 H.264/AVC 인코더 구현을 통하여 제안된 각각의 저전력 IP들이 효율적으로 H.264/AVC 인코더 SoC에서 사용될 수 있음을 확인하였다. In this paper, we are implemented low-power structure for Inter prediction, Intra prediction, Deblocking filter, Transform and Quantization blocks in H.264/AVC Encoder. The proposed Inter/Intra prediction blocks are shown 60.2% cell area reduction by adder reduction through Distributed Arithmetic, 44.3% add operation reduction using MUX for hardware share in Deblocking filter block. Furthermore we applied CSD and CSS process to reduce the cell area instead of multipliers that take a lot of area. The FPGA(Field Programmable Gate Array) and ARM process based H.264/AVC encoder is implemented using proposed low-power IPs. The proposed structure Platforms are implemented to interlock with FPGA and ARM processors. H.264/AVC Encoder implementation using Platforms shows that proposed low-power IPs can use H.264/AVC Encoder SoC effectively.

H.264/AVC에서 향상된 인트라/인터 예측을 위한 모드 추정 방법

박경석(Park, Kyung-Seok),김민준(Kim, Min-Jun),전재현(Jun, Jae-Hyun),류상률(Ryu, Sang-Ryul),김승호(Kim, Seung-Ho) 한국산학기술학회 2012 한국산학기술학회논문지 Vol.13 No.4

H.264/AVC에서는 인트라 예측과 인터 예측의 움직임 추정은 전체 압축 시간의 70 ~ 80%를 차지하게 된다. 다양한 부호화 기술을 이용하여 압축 효율은 높아지지만 복잡도가 증가하여 부호화 시간이 증가하게 되었다. 따라서 본 논문에서는 H.264 부호화 과정 중 화질 손실을 최소한으로 줄이면서 소모되는 시간을 단축시키는 인트라 예측의 블록크기 결정방법과 모드 결정방법을 제안하고자 한다. 추가적으로 계산량을 줄이기 위해 인터 예측의 움직임 추정 을 위해 적응적으로 탐색방법을 결정하는 알고리즘을 제시하고자 한다. 실험을 통해 화질의 열화와 계산량을 측정하 기 위해서 PSNR과 인트라 예측 및 인터 예측 계산 시 소모되는 시간을 알아보았다. 그 결과, 3가지 방법을 모두 사 용 했을 때, 기존의 H.264방법과 비교하여 화질은 거의 비슷하게 유지하면서 실험한 모든 영상의 경우 모든 프레임에 서 평균 500 ~ 600ms정도의 부호화 시간이 단축됨을 보였다. The intra prediction and the motion estimation of inter prediction occupy 70 ~ 80% of whole compression time in H.264/AVC. Compression efficiency has been higher, but complexity has increased and coding time has also increased much more. This paper proposes a block size decision method of the intra prediction and mode decision method which minimize the loss of video quality during the encoding and shorten the time spent. This paper, in addition, proposes an algorithm which determines the method of adaptive block mode for motion estimation of inter prediction. We investigated PSNR and the intra prediction and inter prediction of time-consuming calculations in order to measure video quality degradation and complexity through experiments. Consequently, when you use all three methods, these methods showed that average coding time is shortened as about 500 to 600ms in every frame in the case of all experimented videos, keeping video quality nearly similar, compared with existing methods of H.264.

스케일러블 동영상 부호화 표준에서 움직임 예측 플래그를 위한 효율적인 부호화 방식

문용호,엄일규,하석운 대한임베디드공학회 2014 대한임베디드공학회논문지 Vol.9 No.2

In the scalable video coding standard, inter-layer prediction based on the coding information of the base layer was adopted to increase the coding performance. This prediction tool results in new syntax elements called motion_prediction_flag (mPF) and residul_prediction_ flag(rPF), which are carried to notify the motion vector predictor (MVP) and reference block required in the motion compensation of the decoder. In this paper, an efficient coding method for mPF is proposed to enhance coding efficiency of the salable video coding standard. Through an analysis on the transmission of mPF based on the relationship between the MVPs, we discover the conditions where mPF is unnecessary at the decoder and suggest a modified rate-distortion (RD) cost function to make RD optimization more effective. Simulation results show that the proposed method offers BD rate savings of approximately 1.4%, compared with the conventional SVC standard.

CNN-based Fast Split Mode Decision Algorithm for Versatile Video Coding (VVC) Inter Prediction

Yeo, Woon-Ha,Kim, Byung-Gyu Korea Multimedia Society 2021 The journal of multimedia information system Vol.8 No.3

Versatile Video Coding (VVC) is the latest video coding standard developed by Joint Video Exploration Team (JVET). In VVC, the quadtree plus multi-type tree (QT+MTT) structure of coding unit (CU) partition is adopted, and its computational complexity is considerably high due to the brute-force search for recursive rate-distortion (RD) optimization. In this paper, we aim to reduce the time complexity of inter-picture prediction mode since the inter prediction accounts for a large portion of the total encoding time. The problem can be defined as classifying the split mode of each CU. To classify the split mode effectively, a novel convolutional neural network (CNN) called multi-level tree (MLT-CNN) architecture is introduced. For boosting classification performance, we utilize additional information including inter-picture information while training the CNN. The overall algorithm including the MLT-CNN inference process is implemented on VVC Test Model (VTM) 11.0. The CUs of size 128×128 can be the inputs of the CNN. The sequences are encoded at the random access (RA) configuration with five QP values {22, 27, 32, 37, 42}. The experimental results show that the proposed algorithm can reduce the computational complexity by 11.53% on average, and 26.14% for the maximum with an average 1.01% of the increase in Bjøntegaard delta bit rate (BDBR). Especially, the proposed method shows higher performance on the sequences of the A and B classes, reducing 9.81%~26.14% of encoding time with 0.95%~3.28% of the BDBR increase.

인접블록의 상관관계에 기반한 RGB video coding 개선 알고리즘

김양수(Yang-Soo Kim),정진우(Jin-Woo Jeong),최윤식(Yoon-Sik Choe) 대한전기학회 2009 전기학회논문지 Vol.58 No.12

H.264/AVC High 4:4:4 Intra/Predictive profiles supports RGB 4:4:4 sequences for high fidelity video. RGB color planes rather than YCbCr color planes are preferred by high-fidelity video applications such as digital cinema, medical imaging, and UHDTV. Several RGB coding tools have therefore been developed to improve the coding efficiency of RGB video. In this paper, we propose a new method to extract more accurate correlation parameters for inter-plane prediction. We use a searching method to determine the matched macroblock (MB) that has a similar inter-color relation to the current MB. Using this block, we can infer more accurate correlation parameters to predict chroma MB from luma MB. Our proposed inter-plane prediction mode shows an average bits saving of 15.6% and a PSNR increase of 0.99 ㏈ compared with H.264 high4:4:4 intra-profile RGB coding. Furthermore, extensive performance evaluation revealed that our proposed algorithm has better coding efficiency than existing algorithms..

Novel Motion and Disparity Prediction for Multi-view Video Coding

Lim, Woong,Nam, Junghak,Sim, Donggyu The Institute of Electronics and Information Engin 2014 IEIE Transactions on Smart Processing & Computing Vol.3 No.3

This paper presents an efficient motion and disparity prediction method for multi-view video coding based on the high efficient video coding (HEVC) standard. The proposed method exploits inter-view candidates for effective prediction of the motion or disparity vector to be coded. The inter-view candidates include not only the motion vectors of adjacent views, but also global disparities across views. The motion vectors coded earlier in an adjacent view were found to be helpful in predicting the current motion vector to reduce the number of bits used in the motion vector information. In addition, the proposed disparity prediction using the global disparity method was found to be effective for interview predictions. A multi-view version based on HEVC was used to evaluate the proposed algorithm, and the proposed correspondence prediction method was implemented on a multi-view platform based on HEVC. The proposed algorithm yielded a coding gain of approximately 2.9% in a high efficiency configuration random access mode.