Learning Rich Representations for Robot State Estimation

Abstract

Mobile robots, such as self-driving vehicles, need to operate safely and autonomously in a wide range of environments. This requires an autonomy software stack with robustness built into it at every level. At the root of every decision that a robot makes is its ego-state estimation: establishing where the robot is, how its actuators are positioned, and the related dynamics. As the scale and complexity of real-world robotic systems continue to grow, many research questions remain open: How do we design state estimation systems that are performant, interpretable, and scalable? How sensitive are complex robotic systems to state estimation failures? Can we mirror recent trends in the broader field of deep learning and improve the performance of robot localization by learning on large datasets? This thesis encompasses a line of work which analyzes these questions by studying the robustness of robotic systems at the state estimation level through the lens of robot localization. In the first part of the thesis, we propose a way to formulate localization as an online histogram filtering problem and study the importance of learning task-specific representations and data-driven compression for interpretability and scalability, respectively. In the second part of the thesis, we study state estimation from a systems perspective by analyzing the impact of localization failures on downstream tasks and propose a perception architecture that improves resilience to this family of errors. We continue our analysis by focusing on the impact of large datasets on localization and present Pit30M, a new large-scale localization dataset that helps us draw novel insights into global localization thanks to its highly accurate ground truth. The approaches discussed in this thesis provide insights into how to build accurate and resilient state estimation systems, how to evaluate them holistically in terms of their impact on overall system performance, and how to design localization and mapping systems that scale effortlessly to nation-sized maps.