딥러닝을 활용한 비전 기반 궤적 예측 연구 동향 분석 = Research Trends Analysis of Vision-based Trajectory Prediction Using Deep Learning

다국어 초록 (Multilingual Abstract)

Predicting the movement of objects, such as people, bicycles, and cars, is one of the important technologies. Since the moving object has an attribute value called the time series, technology of predict the trajectory of the moving object is being used in a wide range of fields. The vision-based trajectory prediction model, which predicts the next movement of a vehicle or collision between moving objects with image data from drones and CCTVs that transmit image data in real time, is an important application with the development of image deep learning technology. In this study, we introduced deep learning methodology related with vision-based prediction techniques, which have been studied recently, by dividing in to RNN, feedforward, GAN, attention module, and multimodal information. And also we looked into the field of the applications and the data sets that related with five divided categories. Referring to the methodology, application fields, and available datasets introduced in this study, it is expected to be of practical help in various domestic research fields utilizing the trajectory prediction in the future.

참고문헌 (Reference)

1 박선례 ; 한형석 ; 이경복 ; 사정환 ; 김철홍 ; 임경태, "열차 내 이상행동 검출을 위한 인공지능 엣지 시스템에 대한 연구" 한국철도학회 24 (24): 1062-1074, 2021

2 전인우 ; 이민혁 ; 전철민, "스마트카드 자료를 활용한 대중교통 승객의 통행목적 추정" 한국지리정보학회 22 (22): 28-38, 2019

3 김관호 ; 오규협 ; 이영규 ; 정재윤, "스마트카드 빅데이터를 이용한 서울시 지하철 이동패턴 분석" 한국전자거래학회 18 (18): 211-222, 2013

4 문현구 ; 오규협 ; 김상국 ; 정재윤, "스마트카드 빅데이터를 이용한 서울시 지역별 대중교통 이동 편의성 분석" 대한산업공학회 42 (42): 296-303, 2016

5 최도진 ; 임종태 ; 유승훈 ; 복경수 ; 유재수, "대용량 위치 데이터에서 효율적인 k-최근접 질의 처리 기법" 한국콘텐츠학회 17 (17): 619-630, 2017

6 김종환 ; 이석준 ; 김인철, "다음 장소 예측을 위한 맵리듀스 기반의 이동 패턴 마이닝 시스템 설계" 한국정보처리학회 3 (3): 321-328, 2014

7 Kosaraju, V., "of Advances in Neural Information Processing Systems 32" 2019

8 Zhao, H., "Where are you heading? dynamic trajectory prediction with expert goal examples" 7629-7638, 2021

9 Zeng, K. H., "Visual forecasting by imitating dynamics in natural sequences" 2999-3008, 2017

10 Guo, H., "Vehicle Trajectory Prediction Method Coupled With Ego Vehicle Motion Trend Under Dual Attention Mechanism" 71 : 1-16, 2022

11 Jin, B., "Varnet: Exploring variations for unsupervised video prediction" IEEE 5801-5806, 2018

12 Fu, R., "Using LSTM and GRU neural network methods for traffic flow prediction" IEEE 324-328, 2016

13 Zhong, Y., "Unsupervised learning for forecasting action representations" IEEE 1073-1077, 2018

14 Albawi, S., "Understanding of a convolutional neural network" Ieee 1-6, 2017

15 김혜진, "Travel mode classification method based on travel track information" 한국컴퓨터정보학회 26 (26): 133-142, 2021

16 Walker, J., "The pose knows: Video forecasting by generating pose futures" 3332-3341, 2017

17 Liang, J., "The garden of forking paths:Towards multi-future trajectory prediction" 10508-10518, 2020

18 Gui, L. Y., "Teaching robots to predict human motion" IEEE 562-567, 2018

19 Xu, J., "Structure preserving video prediction" 1460-1469, 2018

20 Jain, A., "Structural-rnn: Deep learning on spatiotemporal graphs" 5308-5317, 2016

21 Zhu, Y., "Starnet: Pedestrian trajectory prediction using deep neural network in star topology" IEEE 8075-8080, 2019

22 Zhang, P., "Sr-lstm: State refinement for lstm towards pedestrian trajectory prediction" 12085-12094, 2019

23 Sadeghian, A., "Sophie:An attentive gan for predicting paths compliant to social and physical constraints" 1349-1358, 2019

24 Alahi, A., "Social lstm:Human trajectory prediction in crowded spaces" 961-971, 2016

25 Gupta, A., "Social gan: Socially acceptable trajectories with generative adversarial networks" 2255-2264, 2018

26 Bock, J., "Self-learning Trajectory Prediction with Recurrent Neural Networks at Intelligent Intersections" 346-351, 2017

27 Xue, H., "SS-LSTM: A hierarchical LSTM model for pedestrian trajectory prediction" IEEE 1186-1194, 2018

28 Hong, J., "Rules of the road: Predicting driving behavior with a convolutional model of semantic interactions" 8454-8462, 2019

29 Ye, X., "Rope3D: The Roadside Perception Dataset for Autonomous Driving and Monocular 3D Object Detection Task" 21341-21350, 2022

30 Furnari, A., "Rollingunrolling lstms for action anticipation from firstperson video" 43 (43): 4021-4036, 2020

31 Sun, C., "Relational action forecasting" 273-283, 2019

32 Gao, J., "Red : Reinforced encoder-decoder networks for action anticipation"

33 Fragkiadaki, K., "Recurrent network models for human dynamics" 4346-4354, 2015

34 Kataoka, H., "Recognition of transitional action for short-term action prediction using discriminative temporal CNN Feature" 1-12, 2016

35 Kabir, M. F., "Real-time vehicular accident prevention system using deep learning architecture" 206 : 117837-, 2022

36 Saleh, K., "Real-time intent prediction of pedestrians for autonomous ground vehicles via spatio-temporal densenet" IEEE 9704-9710, 2019

37 Rhinehart, N., "R2p2: A reparameterized pushforward policy for diverse, precise generative path forecasting" 772-788, 2018

38 Bhattacharjee, P., "Predicting video frames using feature based locally guided objectives" Springer 679-695, 2018

39 Ding, W., "Predicting vehicle behaviors over an extended horizon using behavior interaction network" IEEE 8634-8640, 2019

40 Rhinehart, N., "Precog: Prediction conditioned on goals in visual multi-agent settings" 2821-2830, 2019

41 Tang, J., "Pose guided global and local gan for appearance preserving human video prediction" IEEE 614-618, 2019

42 Rasouli, A., "Pie: A large-scale dataset and models for pedestrian intention estimation and trajectory predicti" 6262-6271, 2019

43 Liang, J., "Peeking into the future:Predicting future person activities and locations in videos" 5725-5734, 2019

44 Zamboni, S., "Pedestrian trajectory prediction with convolutional neural networks" 121 : 108252-, 2022

45 Rehder, E., "Pedestrian prediction by planning using deep neural networks" IEEE 5903-5908, 2018

46 Dominguez-Sanchez, A., "Pedestrian movement direction recognition using convolutional neural networks" 18 (18): 3540-3548, 2017

47 Neumann, L., "Pedestrian and ego-vehicle trajectory prediction from monocular camera" 10204-10212, 2021

48 Rasouli, A., "Pedestrian action anticipation using contextual feature fusion in stacked rnns"

49 Wang, J., "Order matters : Shuffling sequence generation for video prediction"

50 Ding, W., "Online vehicle trajectory prediction using policy anticipation network and optimization-based context reasoning" IEEE 9610-9616, 2019

51 Martinez, J., "On human motion prediction using recurrent neural networks" 2891-2900, 2017

52 Zhou, W., "Neural Computing and Applications" 1-14, 2022

53 Hasan, I., "Mx-lstm: mixing tracklets and vislets to jointly forecast trajectories and head poses" 6067-6076, 2018

54 Lee, J., "Mutual suppression network for video prediction using disentangled features"

55 Yao, T., "Multiple granularity group interaction prediction" 2246-2254, 2018

56 Zhao, T., "Multiagent tensor fusion for contextual trajectory prediction" 12126-12134, 2019

57 Nalcakan, Y., "Monocular Vision-based Prediction of Cut-in Maneuvers with LSTM Networks"

58 Tripicchio, P., "Modeling multiple vehicle interaction constraints for behavior prediction of vehicles on highways" 98 : 107700-, 2022

59 Dendorfer, P., "Mg-gan: A multi-generator model preventing out-ofdistribution samples in pedestrian trajectory prediction" 13158-13167, 2021

60 Choi, C., "Looking to relations for future trajectory forecast" 921-930, 2019

61 Hochreiter, S., "Long short-term memory" 9 (9): 1735-1780, 1997

62 Girase, H., "Loki: Long term and key intentions for trajectory prediction" 9803-9812, 2021

63 Xue, H., "Location-velocity attention for pedestrian trajectory prediction" IEEE 2038-2047, 2019

64 Villegas, R., "Learning to generate long-term future via hierarchical prediction" 3560-3569, 2017

65 Surís, D., "Learning the predictability of the future" 12607-12617, 2021

66 Cho, K., "Learning phrase representations using RNN encoder-decoder for statistical machine translation"

67 Bi, H., "Joint prediction for kinematic trajectories in vehicle-pedestrian-mixed scenes" 10383-10392, 2019

68 Li, J., "Interactionaware multi-agent tracking and probabilistic behavior prediction via adversarial learning" IEEE 6658-6664, 2019

69 Casas, S., "Intentnet:Learning to predict intention from raw sensor data" 947-956, 2018

70 Chen, X., "Intention-aware vehicle trajectory prediction based on spatial-temporal dynamic attention network for internet of vehicles" IEEE 23 (23): 19471-19483, 2022

71 Srikanth, S., "Infer:Intermediate representations for future prediction" IEEE 942-949, 2019

72 Castrejon, L., "Improved conditional vrnns for video prediction" 7608-7617, 2019

73 Wang, Z., "Image quality assessment:from error visibility to structural similarity" 13 (13): 600-612, 2004

74 Ionescu, C., "Human3. 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments" 36 (36): 1325-1339, 2013

75 Villegas, R., "Hierarchical long-term video prediction without supervision" 6038-6046, 2018

76 Veličković, P., "Graph attention networks"

77 Goodfellow, I., "Generative adversarial networks" 63 (63): 139-144, 2020

78 Fernando, T., "Gd-gan: Generative adversarial networks for trajectory prediction and group detection in crowds" Springer 314-330, 2018

79 Ke, Q., "Future moment assessment for action query" 3219-3228, 2021

80 Manglik, A., "Forecasting time-to-collision from monocular video: Feasibility, dataset, and challenges" IEEE 8081-8088, 2019

81 Chao, Y. W., "Forecasting human dynamics from static images" 548-556, 2017

82 Oliu, M., "Folded recurrent neural networks for future video prediction" 716-731, 2018

83 Lu, C., "Flexible spatio-temporal networks for video prediction" 6523-6531, 2017

84 Gui, L. Y., "Few-shot human motion prediction via meta-learning" 432-450, 2018

85 Bhattacharyya, A., "Euro-pvi: Pedestrian vehicle interactions in dense urban centers" 6408-6417, 2021

86 Xu, Y., "Encoding crowd interaction with deep neural network for pedestrian trajectory prediction" 5275-5284, 2018

87 Yuan, Y., "Ego-pose estimation and forecasting as real-time pd control" 10082-10092, 2019

88 Liu, W., "Dyan: A dynamical atoms-based network for video prediction" 170-185, 2018

89 Liang, X., "Dual motion GAN for future-flow embedded video prediction" 1744-1752, 2017

90 Narayanan, S., "Divide-and-conquer for laneaware diverse trajectory prediction" 15799-15808, 2021

91 Lee, N., "Desire: Distant future prediction in dynamic scenes with interacting agents" 336-345, 2017

92 Katariya, V., "Deeptrack : lightweight deep learning for vehicle trajectory prediction in highways" 23 (23): 18927-18936, 2022

93 Cai, H., "Deep video generation, prediction and completion of human action sequences" 366-382, 2018

94 Ho, Y. H., "Deep reinforcement learning for video prediction" IEEE 604-608, 2019

95 Strickland, M., "Deep predictive models for collision risk assessment in autonomous driving" IEEE 4685-4692, 2018

96 Karishma Pawar ; Vahida Attar, "Deep learning based detection and localization of road accidents from traffic surveillance videos" 한국통신학회 8 (8): 379-387, 2022

97 Hui, F., "Deep encoder–decoder-NN: A deep learning-based autonomous vehicle trajectory prediction and correction model" 593 : 126869-, 2022

98 Byeon, W., "Contextvp: Fully contextaware video prediction" 753-769, 2018

99 Bartoli, F., "Context-aware trajectory prediction" IEEE 1941-1946, 2018

100 Li, J., "Conditional generative neural system for probabilistic trajectory prediction" IEEE 6150-6156, 2019

101 Ye, Y., "Compositional video prediction" 10353-10362, 2019

102 Konakalla, N., "Cnn, cnn encoder-rnn decoder, and pretrained vision transformers for surrounding vehicle lane change classification at future time steps" 2022

103 Gujjar, P. Vaughan, R., "Classifying pedestrian actions in advance using predicted video of urban driving scenes" IEEE 2097-2103, 2019

104 Chen, L., "CAE‐GAN: A hybrid model for vehicle trajectory prediction" 16 (16): 1682-1696, 2022

105 Rasouli, A., "Bifold and semantic reasoning for pedestrian behavior prediction" 15600-15610, 2021

106 Ying, G., "Better guider predicts future better: Difference guided generative adversarial networks" Springer 277-292, 2018

107 Kotseruba, I., "Benchmark for evaluating pedestrian action prediction" 1258-1268, 2021

108 Gao, J., "Attentionbased global context network for driving maneuvers prediction" 33 (33): 1-11, 2022

109 Scheel, O., "Attention-based lane change prediction" IEEE 8655-8661, 2019

110 Rasouli, A., "Are they going to cross? a benchmark dataset and baseline for pedestrian crosswalk behavior" 206-213, 2017

111 Girdhar, R., "Anticipative video transformer" 13505-13515, 2021

112 Vondrick, C., "Anticipating visual representations from unlabeled video" 98-106, 2016

113 Suzuki, T., "Anticipating traffic accidents with adaptive loss and large-scale incident db" 3521-3529, 2018

114 Bütepage, J., "Anticipating many futures: Online human motion prediction and generation for human-robot interaction" IEEE 4563-4570, 2018

115 Shi, K., "An integrated car-following and lane changing vehicle trajectory prediction algorithm based on a deep neural network" 599 : 127303-, 2022

116 Lv, Z., "An improved GAN with transformers for pedestrian trajectory prediction models" 37 (37): 4417-4436, 2022

117 Zeng, K. H., "Agent-centric risk assessment: Accident anticipation and risky region localization" 2222-2230, 2017

118 Gui, L. Y., "Adversarial geometry-aware human motion prediction" 786-803, 2018

119 Kim, Y., "Advances in Neural Information Processing Systems 32" 2019

120 Xu, J., "Advances in Neural Information Processing Systems 31" 2018

121 Hsieh, J. T., "Advances in Neural Information Processing Systems 31" 2018

122 Bhattacharjee, P., "Advances in Neural Information Processing Systems 30" 2017

123 Wang, Y., "Advances in Neural Information Processing Systems 30" 2017

124 Finn, C., "Advances in Neural Information Processing Systems 29" 2016

125 Chiu, H. K., "Action-agnostic human pose forecasting" IEEE 1423-1432, 2019

126 Lee, H., "A study on combine artificial intelligence models for multiclassification for an abnormal behaviors in cctv images" 26 (26): 498-500, 2022

127 Kim, T. K., "A research on the fainting monitoring system using deep learning" 237-238, 2021

128 Gopalakrishnan, A., "A neural temporal model for human motion prediction" 12116-12125, 2019

129 Pfeiffer, M., "A data-driven model for interaction-aware pedestrian motion prediction in object cluttered environments" IEEE 5921-5928, 2018

130 Inturi, A. R., "A Novel Vision-Based Fall Detection Scheme Using Keypoints of Human Skeleton with Long Short-Term Memory Network" 1-13, 2022