Context and Behavioral Analysis for Pedestrians in the Domain of Self-Driving [electronic resource]

다국어 초록 (Multilingual Abstract)

Autonomous Ground Vehicles (AGVs) have made their way into various industries, including transportation, delivery services, healthcare, and logistics handling. In order to thrive in these sectors, AGVs must possess the ability to accurately detect, track, anticipate, and navigate around pedestrians in their surroundings. Over time, a multitude of new tracking and forecasting algorithms have been developed to enhance the effectiveness and safety of AGVs. This progress has been largely facilitated by the availability of public datasets and challenges centered around self-driving technologies.However, it is important to note that most of these datasets and approaches primarily focus on cars, often neglecting the evaluation of pedestrian performance. This is primarily due to the limited number of pedestrian annotations compared to vehicle annotations within these datasets.In this dissertation, I present a series of studies aimed at enhancing the performance of pedestrian tracking, forecasting, and anomaly detection. To begin, I introduce a novel self-driving dataset designed to achieve a more balanced representation of pedestrians. The dataset captures data along a 15 km route, encompassing diverse scenes such as urban, highway, rural, and campus environments. It also contains various weather conditions such as snow, rain, and sun and incorporates different time periods, including day and night. Furthermore, the dataset covers a wide range of traffic conditions and compositions, including pedestrians, cyclists, and cars. To facilitate the training of machine learning models, the dataset includes road and object annotations utilizing amodal masks to capture partial occlusions and 3D bounding boxes effectively.Secondly, I delve into my research on priority tracking of pedestrians in self-driving scenarios. This work capitalizes on the concept of reachability to determine the priority of pedestrians based on safety considerations. This framework ensures efficient resource allocation and optimized tracking performance by employing computationally intensive tracking algorithms on agents that pose potential safety risks and utilizing computationally lightweight tracking approaches for pedestrians who do not present such concerns.Next, I introduce a novel prediction framework that capitalizes on pedestrian-focused contextual cues. Our approach involves an encoder-decoder Long-short-term memory (LSTM) network, which effectively encodes past behaviors, sequential appearance, pose features, and map context. By leveraging these elements, we can accurately forecast the future trajectory of a target pedestrian.Finally, I present my research on anomaly detection of pedestrians in the context of self-driving cars. We propose a framework for generating weak labels using raw data and repeated traversals, thereby alleviating the need for laborious and costly manual annotation of pedestrian data. This approach has the potential to enhance the training of algorithms specifically tailored to pedestrian-related tasks. We then introduce a probabilistic framework and train it to learn the typical behavior patterns of pedestrians utilizing weak labels. Lastly, we demonstrate the framework's qualitative ability to identify anomalies within a given scene.