LVLN: 시각-언어 이동을 위한 랜드마크 기반의 심층 신경망 모델 = LVLN: A Landmark-Based Deep Neural Network Model for Vision-and-Language Navigation

다국어 초록 (Multilingual Abstract)

In this paper, we propose a novel deep neural network model for Vision-and-Language Navigation (VLN) named LVLN (Landmark- based VLN). In addition to both visual features extracted from input images and linguistic features extracted from the natural language instructions, this model makes use of information about places and landmark objects detected from images. The model also applies a context-based attention mechanism in order to associate each entity mentioned in the instruction, the corresponding region of interest (ROI) in the image, and the corresponding place and landmark object detected from the image with each other. Moreover, in order to improve the success rate of arriving the target goal, the model adopts a progress monitor module for checking substantial approach to the target goal. Conducting experiments with the Matterport3D simulator and the Room-to-Room (R2R) benchmark dataset, we demonstrate high performance of the proposed model.

국문 초록 (Abstract)

본 논문에서는 시각-언어 이동 문제를 위한 새로운 심층 신경망 모델인 LVLN을 제안한다. LVLN 모델에서는 자연어 지시의 언어적 특징과 입력 영상 전체의 시각적 특징들 외에, 자연어 지시에서 언급하는 주요 장소와 랜드마크 물체들을 입력 영상에서 탐지해내고 이 정보들을 추가적으로 이용한다. 또한 이 모델은 자연어 지시 내 각 개체와 영상 내 각 관심 영역, 그리고 영상에서 탐지된 개별 물체 및 장소 간의 서로 연관성을 높일 수 있도록 맥락 정보 기반의 주의 집중 메커니즘을 이용한다. 그뿐만 아니라, LVLN 모델은 에이전트의 목표 도달 성공율을 향상시키기 위해, 목표를 향한 실질적인 접근을 점검할 수 있는 진척 점검기 모듈도 포함하고 있다. Matterport3D 시뮬레이터와 Room-to-Room (R2R) 벤치마크 데이터 집합을 이용한 다양한 실험들을 통해, 본 논문에서 제안하는 LVLN 모델의 높은 성능을 확인할 수 있었다.

참고문헌 (Reference)

1 J. Redmon, "You Only Look Once: Unified, Real-Time Object Detection" 2016

2 J. Redmon, "YOLOv3: An Incremental Improvement"

3 J. Redmon, "YOLO9000: Better, Faster, Stronger" 2017

4 A. Das, "Visual Dialog" 2017

5 P. Anderson, "Vision-and-Language Navigation: Interpreting Visually-grounded Navigation Instructions in Real Environments" 2018

6 K. Simonyan, "Very Deep Convolutional Networks for Large-Scale Image Recognition" 2015

7 A. Agrawal, "VQA: Visual Question Answering" 2425-2433, 2015

8 C. Ma, "The Regretful Agent:Heuristic-Aided Navigation through Progress Estimation" 2019

9 L. Ke, "Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation" 2019

10 W. Liu, "Ssd: Single Shot Multibox Detector" Springer 21-37, 2016

11 D. Fried, "Speaker-Follower Models for Vision-and-Language Navigation" 28 : 2018

12 C. Ma, "Self-Monitoring Navigation Agent via Auxiliary Progress Estimation" 2019

13 R. Grishick, "Rich Feature Hierarchies for Accurate Oobject Detection and Semantic Segmentation" 2014

14 X. Wang, "Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation" 2019

15 B. Zhou, "Places: A 10million Image Database for Scene Recognition" 40 : 1452-1464, 2017

16 T.-Y. Lin, "Microsoft COCO: Common Objects in Context" 13 : 740-755, 2014

17 A. Chang, "Matterport3D:Learning from RGB-D Data in Indoor Environments" 5 : 2017

18 K. He, "Mask R-CNN" 2017

19 X. Wang, "Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigation" 696-711, 2018

20 N. Silberman, "Indoor Segmentation and Support Inference from RGBD Images" 746-760, 2012

21 J. Deng, "ImageNet:A Large-Scale Hierarchical Image Database" 2009

22 D. Gordon, "IQA:Visual Question Answering in Interactive Environments" 2018

23 C. Szegedy, "Going Deeper with Convolutions" 1-9, 2015

24 S. Ren, "Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks" 2015

25 R. Girshick, "Fast R-CNN" 2015

26 A. Das, "Embodied Question Answering" 5 : 2018

27 A. Hanni, "Deep learning Framework for Scene based Indoor Location Recognition" IEEE 2017

28 K. He, "Deep Residual Learning for Image Recognition" 770-778, 2016

29 K. Wang, "A Discriminative Algorithm for Indoor Place Recognition based on Clustering of Features and Images" 14 : 407-419, 2017