Camera Mounting Invariant Object Detection using Novel View Augmentation

Background: Object detection is a critical component of Autonomous Driving Systems (ADS) and Advanced Driver Assistance Systems (ADAS), requiring robust performance across varying camera viewpoints. Traditional data collection methods are often constrained by the high costs and complexities associated with capturing diverse scenarios, making novel view synthesis (NVS) a promising alternative for data augmentation. This thesis explores the use of NVS to enhance object detection models by generating synthetic views from existing images. Objectives: This research aims to implement two novel view synthesis techniques for data augmentation, focusing on generating images with variations in height and pitch. These techniques will be evaluated and compared to identify the superior method based on reconstruction quality metrics. Finally, the effectiveness of the methods will be assessed by applying an object detection algorithm to various dataset combinations, determining the best technique for enhancing object detection in autonomous driving scenarios. Methods: The methodology involves a comparative analysis of two NVS techniques: a MPI-based NVS, which utilizes adaptive multiplane images for rendering novel views, and a point cloud transformation-based approach that manipulates 3D point clouds to simulate different camera perspectives. The best-performing NVS method is then applied to augment training data for a YOLOv8 object detection model. Evaluation metrics such as Peak Signal-to-Noise Ratio (PSNR), Mean Squared Error (MSE), and Structural Similarity Index (SSIM) are used to assess the quality of the synthesized views. Results: The MPI-based NVS method, outperformed the point cloud transformation- based approach in both perceptual and reconstruction quality metrics, such as PSNR, MSE, and SSIM, producing higher fidelity synthetic views. Incorporating NVS- augmented data into the object detection model significantly improved its ability to generalize to unseen viewpoints, enhancing detection accuracy. These results ...