Sound Source Localization in Complex Indoor Environment: A Self-Supervised Incremental Learning Approach

Sound source localization is essential in robotics, which broadens the possibilities of human-robot interactions by enriching the robot's perceptual capabilities. Localizing an acoustic source in a complex indoor environment is especially challenging due to the high noise-to-signal ratio and reverberations. In this thesis, we present an incremental learning framework for mobile robots localizing the human sound source using a microphone array in a complex indoor environment consisting of multiple rooms. The framework allows robots to accumulate training data and improve the performance of the prediction model over time using an incremental learning scheme. A self-supervision process is developed such that the model ranks the priority of rooms to explore, assigns the ground truth label to the collected data, and updates the learned model on-the-fly. In experiments, we demonstrate that the framework can be directly deployed in real-world scenarios without extra human interventions, and can localize the sound source successfully.