Video Multiple Classification Algorithm Based on SVM

Chao Jiang,Shuguang Wang 보안공학연구지원센터 2016 International Journal of Signal Processing, Image Vol.9 No.7

The performance of video automatic classification algorithm depends largely on the extraction of video features and selection of classification algorithm. From the perspective of video contents and video style type, the paper presents a new feature representation scheme, i.e. MPEG-7 visual description sub-combination model, a new method based on support vector machine (SVM) to solve problems with existing algorithms, by analyzing visual differences between five types of videos. Also we improve the classifier decision scheme and then propose the secondary prediction mechanism based on SVM 1-1 approach, improving the accuracy of SVM multi-classification method. The experimental results indicate that the proposed method manifests differences of different videos about feature selection, enhances the discrimination ability of videos pending for classification and increases the effectiveness of SVM multi-video classification.

장면 전환의 히스토그램 특징을 이용한 동영상 분류 포렌식

최혜민,김준수,황현욱,이해연 한국디지털포렌식학회 2023 디지털 포렌식 연구 Vol.17 No.4

Due to the recent popularization of multimedia platforms, there is a need for a forensic method to support the illegality analysis of the numerous distributed videos. Recently, research on introducing artificial intelligence technology to image analysis is being actively conducted. This paper proposes a forensic algorithm to select videos for the illegality analysis, which extracts the histogram of frame difference value from an input video to create a visual rhythm and detect shot change using modified DenseNet-201 model to classify the video into edited or unedited video. Edited videos included sudden shot changes and angle changes, and non-edited videos were defined as videos that did not. A video of random length was divided into 3-minute segments, and a visual rhythm image was created with a histogram of frame difference values. This image was applied to the modified DenseNet-201 model to classify into edited and unedited segments, and finally classified edited and unedited videos through threshold filtering for the ratio. Through experiments, it was shown that the proposed video classification forensic algorithm has 95.91% classification accuracy for each segment and 94.50% classification accuracy for the entire video. 멀티미디어 플랫폼의 대중화로 인하여 유통되고 있는 수많은 동영상들의 불법성 분석을 지원할 포렌식 방법이 필요하며, 최근 동영상 분류에 인공지능 기술을 도입하는 연구가 활발히 진행되고 있다. 본 논문에서는 입력된 동영상에서 프레임 차이값 히스토그램을 추출하여 비주얼 리듬을 생성하고 변형된 DenseNet-201 모델을 활용하여 장면 전환을 검출함으로써 편집 및 무편집 동영상으로 분류하여 불법성 분석 대상을 선별하는 포렌식 알고리즘을 제안한다. 편집 동영상은 급격한 장면 전환과 앵글 변화가 포함되었고, 무편집 동영상은 그렇지 않은 동영상으로 정의했다. 무작위 길이의 동영상을 3분 단위의 세그먼트로 분할하고, 프레임 차이값 히스토그램으로 비주얼 리듬 이미지를 생성하였다. 이를 변형된 DenseNet-201 모델에 적용하여 편집 및 무편집 세그먼트로 구분하였고, 비율에 대한 임계값 필터링을 통하여 최종적으로 편집 및 무편집 동영상 분류를 수행했다. 실험을 통하여 제안하는 동영상 분류 포렌식 알고리즘이 세그먼트 단위로 95.91%, 전체 동영상에 대해 94.50% 분류 정확도를 갖고 있음을 보였다.

영상실감을 위한 후각정보에 대한 사용자 지각과 영상분류

이국희(Guk-Hee Lee),이형철(Hyung-Chul O. Li),안충현(Chung Hyun Ahn),최지훈(Ji Hoon Choi),김신우(ShinWoo Kim) 한국HCI학회 2013 한국HCI학회 논문지 Vol.8 No.2

시청각정보를 통한 영상미디어의 실감향상에 대해서는 많은 진보가 이루어져 왔다. 반면 영상실감을 위한 후각정보의 제시는 기술적 구현과 통제가 어려워 관련연구를 찾아보기 힘들다. 본 연구는 영상실감을 위해 후각정보를 제시하고자 할 때 필요한 기초자료를 획득하기 위해, 영상에 존재하는 후각정보에 대한 사용자 지각을 조사한 후 이에 근거하여 다양한 영상을 분류한 것이다. 이를 위해 먼저 영상에 냄새가 존재하는지 (냄새존재여부), 영상과 함께 해당 냄새를 경험하고 싶은지 (냄새제시선호), 해당 냄새가 내가 좋아하는 냄새인지 (냄새자체선호), 해당 냄새가 제시된다면 강도는 어느 정도면 좋을지 (냄새제시강도), 그리고 영상 속의 냄새가 얼마나 구체적인지 (냄새의 구체성)라는 다섯 가지 질문을 선정하였다. 이 질문들에 대해 높은 혹은 낮은 평정을 받을 만한 다양한 장르의 영상을 수집한 다음, 참가자들에게 하나씩 시청하게 한 후 위의 다섯 가지 질문에 대해 7점 척도로 평정하게 하였다. 영상분류를 위해 위 다섯 가지 질문을 2개씩 쌍으로 묶은 후, 각 질문에 대한 평정값을 2차원 평면의 X-Y축으로 설정하여 영상 산포도를 구성하였다. 산포도를 통해 드러난 영상군집과 그 형태는 해당 영상군집이 가진 특성에 대한 통찰을 제공함과 동시에 영상실감을 위한 후각정보 제시에 중요한 시사점을 줄 것으로 기대한다. There has been much advancement in reality enhancement using audio-visual information. On the other hand, there is little research on provision of olfactory information because smell is difficult to implement and control. In order to obtain necessary basic data when intend to provide smell for video reality, in this research, we investigated user perception of smell in diverse videos and then classified the videos based on the collected user perception data. To do so, we chose five main questions which were ‘whether smell is present in the video’(smell presence), ‘whether one desire to experience the smell with the video’(preference for smell presence with the video), ‘whether one likes the smell itself’(preference for the smell itself), ‘desired smell intensity if it is presented with the video’(smell intensity), and ‘the degree of smell concreteness’(smell concreteness). After sampling video clips of various genre which are likely to receive either high and low ratings in the questions, we had participants watch each video after which they provided ratings on 7-point scale for the above five questions. Using the rating data for each video clips, we constructed scatter plots by pairing the five questions and representing the rating scale of each paired questions as X-Y axes in 2 dimensional spaces. The video clusters and distributional shape in the scatter plots would provide important insight into characteristics of each video clusters and about how to present olfactory information for video reality.

Extraction of User Preference for Video Stimuli Using EEG-Based User Responses

문진영,김영래,이형직,배창석,윤완철 한국전자통신연구원 2013 ETRI Journal Vol.35 No.6

Owing to the large number of video programs available, a method for accessing preferred videos efficiently through personalized video summaries and clips is needed. The automatic recognition of user states when viewing a video is essential for extracting meaningful video segments. Although there have been many studies on emotion recognition using various user responses, electroencephalogram (EEG)-based research on preference recognition of videos is at its very early stages. This paper proposes classification models based on linear and nonlinear classifiers using EEG features of band power (BP) values and asymmetry scores for four preference classes. As a result, the quadratic-discriminant-analysisbased model using BP features achieves a classification accuracy of 97.39% (±0.73%), and the models based on the other nonlinear classifiers using the BP features achieve an accuracy of over 96%, which is superior to that of previous work only for binary preference classification. The result proves that the proposed approach is sufficient for employment in personalized video segmentation with high accuracy and classification power.

포렌식을 위한 Conv3DNet 기반의 동의 및 비동의 동영상 검출

한현택,김준수,황현욱,이해연 한국디지털포렌식학회 2023 디지털 포렌식 연구 Vol.17 No.3

고도화되는 디지털 범죄를 해결하기 위해 다양한 디지털 포렌식 기술이 연구되고 있다. 특히, 동영상을 이용한 범죄가 증가하고 있지만 방대한 데이터 규모로 인해 인력 및 자원 측면에서 제약이 있어서 동영상 포렌식에 인공지능을 도입하는 연구도 진행되고 있다. 본 논문에서는 Conv3DNet 모델을 이용하여 다양한 동영상에서 상대 동의를 구하고 촬영한 동영상(동의) 및 상대 동의를 구하지 않고 촬영한 동영상(비동의)으로 분류하는 알고리즘을 제안한다. 먼저 동의 및 비동의 동영상의 기준을 정의하고, 장시간의 동영상을 세그먼트 단위로 분할하여 동의 및 비동의 동영상 데이터셋을 구축하였다. 그 후에 세그먼트 단위의 동영상에 대해 프레임 특징을 추출하는 전처리를 수행하였고, 연속적인 프레임들에서 인접한 프레임 내 객체의 외관과 행동을 추적하여 학습하는 Conv3DNet 모델에 적용하였다. 특히, Conv3DNet 모델은 동의 및 비동의 동영상 분류를 위하여 변형과 파라미터 최적화를 수행하였다. 제안한 알고리즘의 성능 검증은 세 종류의 옵티마이저(SGD, Adam, Nadam)를 이용하여 수행하였고, 최종적으로 Nadam 옵티마이저를 이용한 동의 및 비동의 동영상 분류 알고리즘을 통해 94.18% 정확도를 달성하였다. Various digital forensic technologies are being studied to solve increasingly sophisticated digital crimes. In particular, crimes using video are increasing, but, there are limits in terms of manpower and resources due to the massive amount of data. Therefore, research on applying artificial intelligence to video forensics is also being conducted. In this paper, we propose an algorithm for classifying various video files into videos taken after obtaining consent (agreed) and videos filmed without consent (non-agreed) using the Conv3DNet model. First, the criteria of agreed and non-agreed video is defined, and video datasets are constructed by segmenting long-time videos into segments. After that, pre-processing is performed to extract frame features in segment units and applied to the Conv3DNet model, which learns by tracking the appearance and behavior of objects in adjacent frames of successive frames. In particular, the Conv3DNet model is modified and optimized its parameters to classify agreed and non-agreed videos. The performance of the proposed algorithm was verified using three types of optimizers(SGD, Adam, and Nadam), and finally achieved 94.18% accuracy in the agreed and non-agreed video classification using the Nadam optimizer.

Towards Intelligent Road Situation Detection with Deep Learning Approaches

Subhajit Chatterjee,Yung-Cheol Byun 한국정보기술학회 2024 Proceedings of The International Workshop on Futur Vol.2024 No.4

The growing importance of video situation classification lies in its ability to understand and classify road situations. This field has seen significant progress in recent years, fueled by advancements in deep learning models. However, a major challenge remains, existing systems often require extensive video data for training. While this approach can achieve excellent results, it often necessitates pre-training these models on massive datasets a process impractical for applications with limited video data. To address this data-efficiency bottleneck, we employed Video-masked Autoencoders (VideoMAE) as a pre-training technique for road video situation classification. VideoMAE utilizes a self-supervised learning approach, enabling it to learn meaningful representations from unlabeled video data. We evaluate multiple iterations of our VideoMAE model on a road video dataset. Our results demonstrate that VideoMAE achieves impressive performance even with a very limited dataset. This data efficiency makes Video-MAE a compelling solution for real-time road situation classification tasks in resource-constrained environments.

An Optimized e-Lecture Video Search and Indexing framework

Medida, Lakshmi Haritha,Ramani, Kasarapu International Journal of Computer ScienceNetwork S 2021 International journal of computer science and netw Vol.21 No.8

The demand for e-learning through video lectures is rapidly increasing due to its diverse advantages over the traditional learning methods. This led to massive volumes of web-based lecture videos. Indexing and retrieval of a lecture video or a lecture video topic has thus proved to be an exceptionally challenging problem. Many techniques listed by literature were either visual or audio based, but not both. Since the effects of both the visual and audio components are equally important for the content-based indexing and retrieval, the current work is focused on both these components. A framework for automatic topic-based indexing and search depending on the innate content of the lecture videos is presented. The text from the slides is extracted using the proposed Merged Bounding Box (MBB) text detector. The audio component text extraction is done using Google Speech Recognition (GSR) technology. This hybrid approach generates the indexing keywords from the merged transcripts of both the video and audio component extractors. The search within the indexed documents is optimized based on the Naïve Bayes (NB) Classification and K-Means Clustering models. This optimized search retrieves results by searching only the relevant document cluster in the predefined categories and not the whole lecture video corpus. The work is carried out on the dataset generated by assigning categories to the lecture video transcripts gathered from e-learning portals. The performance of search is assessed based on the accuracy and time taken. Further the improved accuracy of the proposed indexing technique is compared with the accepted chain indexing technique.

Classification of TV Program Scenes Based on Audio Information

Lee, Kang-Kyu,Yoon, Won-Jung,Park, Kyu-Sik The Acoustical Society of Korea 2004 韓國音響學會誌 Vol.23 No.e3

In this paper, we propose a classification system of TV program scenes based on audio information. The system classifies the video scene into six categories of commercials, basketball games, football games, news reports, weather forecasts and music videos. Two type of audio feature set are extracted from each audio frame-timbral features and coefficient domain features which result in 58-dimensional feature vector. In order to reduce the computational complexity of the system, 58-dimensional feature set is further optimized to yield l0-dimensional features through Sequential Forward Selection (SFS) method. This down-sized feature set is finally used to train and classify the given TV program scenes using κ -NN, Gaussian pattern matching algorithm. The classification result of 91.6% reported here shows the promising performance of the video scene classification based on the audio information. Finally, the system stability problem corresponding to different query length is investigated.

Blind Sharpness Prediction Based on Image-Based Motion Blur Analysis

Taegeun Oh,Sanghoon Lee [Institute of Electrical and Electronics Engineers 2015 IEEE transactions on broadcasting Vol.61 No.1

<P>For high bit rate video, it is important to acquire the video contents with high resolution, the quality of which may be degraded due to the motion blur from the movement of an object(s) or the camera. However, conventional sharpness assessments are designed to find focal blur caused either by defocusing or by compression distortion targeted for low bit rates. To overcome this limitation, we present a no-reference framework of a visual sharpness assessment (VSA) for high-resolution video based on the motion and scene classification. In the proposed framework, the accuracy of the sharpness estimation can be improved via pooling weighted by the visual perception from the object and camera movements and by the strong influence from the region with the highest sharpness. Based on the motion blur characteristics, the variance and the contrast over the spectral domain are used to quantify the perceived sharpness. Moreover, for the VSA, we extract the highly influential sharper regions and emphasize them by utilizing the scene adaptive pooling. Based on the subjective results, we demonstrate that the VSA can measure the video sharpness more accurately than other sharpness measurements for high-resolution video.</P>

An integrated music video browsing system for personalized television

Kim, H.G.,Kim, J.Y.,Baek, J.G. Pergamon ; Elsevier Science Ltd 2011 expert systems with applications Vol.38 No.1

In this paper, we propose an integrated music video browsing system for personalized digital television. The system has the functions of automatic music emotion classification, automatic theme-based music classification, salient region detection, and shot classification. From audio (music) tracks, highlight detection and emotion classification are performed on the basis of information on temporal energy, timbre and tempo. For video tracks, shot detection is fulfilled to classify shots into face shots and color-based shots. Lastly automatic grouping of themes is executed on music titles and their lyrics. With a database of international music videos, we evaluate the performance of each function implemented in this paper. The experimental results show that the music browsing system achieves remarkable performances. Thus, our system can be adopted in any digital television for providing personalized services.