Predicting to Improve: Integrity Measures for Assessing Visual Localization Performance

Visual Place Recognition (VPR) is a key component of many robot localization and mapping system processing pipelines, providing loop closure and coarse topological localization priors for pose refinement stages. When deploying these systems in the real-world, system self-characterization of when it is performing well or poorly can be more important than absolute performance. In this research, we demonstrate a new supervised learning approach to predicting localization integrity on a frame-by-frame basis along a route, using artefacts from the localization technique itself. Our method involves a lightweight post-processing step that is VPR technique-agnostic; it can be calibrated for any given place recognition technique, such that inaccurate localization points along a route can be identified and discarded. Unlike the normal parameter sweeping evaluation of a VPR system, which directly trades precision for recall, when deployed on a baseline VPR system our approach improves the precision without significantly reducing recall, resulting in improved average localization performance. Over twelve combinations of varied datasets and VPR techniques, we demonstrate our approach is able to predict localization errors with an average precision of 74%, resulting in an improvement in mean localization accuracy.