Continuous Control of Autonomous Vehicles using Plan-assisted Deep Reinforcement Learning

다국어 초록 (Multilingual Abstract)

End-to-end deep reinforcement learning (DRL) is emerging as a promising paradigm for autonomous driving. Although DRL provides an elegant framework to accomplish final goals without extensive manual engineering, capturing plans and behavior using deep neural networks is still an unsolved issue. End-to-end architectures, as a result, are currently limited to simple driving scenarios, often performing sub-optimally when rare, unique conditions are encountered. We propose a novel plan-assisted deep reinforcement learning framework that, along with the typical state-space, leverages a “trajectory-space” to learn optimal control. While the trajectory-space, generated by an external planner, intrinsically captures the agent’s high-level plans, world models are used to understand the dynamics of the environment for learning behavior in latent space. An actor-critic network, trained in imagination, uses these latent features to predict policy and state-value function. Based primarily on DreamerV2 and Racing Dreamer, the proposed model is first trained in a simulator and eventually tested on the F1TENTH race car. We evaluate our model for best lap times against parametertuned and learning-based controllers on unseen race tracks and demonstrate that it generalizes to complex scenarios where other approaches perform sub-optimally. Furthermore, we show the model’s enhanced stability as a trajectory tracker and establish the improvement in interpretability achieved by the proposed framework.

목차 (Table of Contents)

• Abstract
• 1. INTRODUCTION
• 2. RELATED WORKS
• 3. PRELIMINARIES
• 4. PLAN-ASSISTED REINFORCEMENT LEARNING
• 5. MODEL ARCHITECTURE AND TRAINING
• 6. EXPERIMENTS & RESULTS
• 7. CONCLUSION
• REFERENCES