Advancing Gesture Recognition with Millimeter Wave Radar

Wireless sensing has attracted significant interest over the years, and with the dawn of emerging technologies, it has become more integrated into our daily lives. Among the various wireless communication platforms, WiFi has gained widespread deployment in indoor settings. Consequently, the utilization of ubiquitous WiFi signals for detecting indoor human activities has garnered considerable attention in the past decade. However, more recently, mmWave Radar-based sensing has emerged as a promising alternative, offering advantages such as enhanced sensitivity to motion and increased bandwidth. This thesis introduces innovative approaches to enhance contactless gesture recognition by leveraging emerging low-cost millimeter wave radar technology. It makes three key contributions. Firstly, a cross-modality training technique is proposed, using mmWave radar as a supplementary aid for training WiFi-based deep learning models. The proposed model enables precise gesture detection based solely on WiFi signals, significantly improving WiFi-based recognition. Secondly, a novel beamforming-based gesture detection system is presented, utilizing commodity mmWave radars for accurate detection in low signal-to-noise scenarios. By steering multiple beams around the gesture performer, independent views of the gesture are captured. A selfattention based deep neural network intelligently fuses information from these beams, surpassing single-beam accuracy. The model incorporates a unique data augmentation algorithm accounting for Doppler shift and multipath effects, enhancing generalization. Notably, the proposed method achieves superior gesture classification performance, outperforming state-of-the-art approaches by 31-43% with only two beams. Thirdly, the research explores receiver antenna diversity in mmWave radars to further improve gesture recognition accuracy by deep learning techniques to combine data from multiple receiver antennas, leveraging inherent diversity for enhanced detection. Extensive experimentation and ...