Fusion of SAR and Electrooptical Imagery for Vehicle Classification Using Neural Networks
Open Access
- Author:
- Wood, Noah
- Graduate Program:
- Electrical Engineering
- Degree:
- Master of Science
- Document Type:
- Master Thesis
- Date of Defense:
- May 09, 2022
- Committee Members:
- Kultegin Aydin, Program Head/Chair
Ram Mohan Narayanan, Thesis Advisor/Co-Advisor
David Marion Jenkins, Jr., Committee Member - Keywords:
- machine learning
data fusion
sar
variability - Abstract:
- Within the field of automatic target recognition, significant attention is given to data fusion techniques to optimize decision making in systems of multiple sensors. The challenge of fusing synthetic aperture radar (SAR) and electrooptical (EO) imagery is of particular interest to the defense community due to those sensors’ prevalence in target recognition systems. In this thesis, we compare the performances of two network architectures (a simple six-layer CNN and an 18-layer ResNet), each implemented with multiple fusion methods trained to classify SAR and EO imagery of military targets. The Synthetic and Measured Paired Labeled Experiment (SAMPLE) dataset is used, which is an expansion of the MSTAR dataset, using both original measured SAR data and synthetic EO data. This work compares the classification performance of both networks using the data modalities individually, using feature level fusion, using decision level fusion, and using a novel fusion method based on the three RGB-input channels of the ResNet (or other CNN for color image processing). In the input channel fusion method proposed, SAR imagery is fed to one of the three input channels, EO data is passed to a second of the three input channels, and a zero vector given to the third channel (which could be replaced by a third sensor, if available). Despite its simplicity and off-the-shelf implementation, the input channel fusion method provides strong results that are worthy of study. To illustrate the variability of the machine learning algorithms each time they are run, many iterations of each experiment are collected. The distributions of final averages are presented in the form of violin plots, averaged confusion matrices, and confusion variance and standard deviation matrices.