|
This study wishes to apply a non-parametric estimate of the Bayes Error Rate (BER) to current Air Force problems related to target classification. Whether they be neural networks, autoencoders, or other architectures, classifiers are commonly assessed through confusion matrices and associated statistics, or visualizations of feature spaces like t-SNE plots. However, these methods depend on the test data used to assess the performance of the network, not the robustness of the classifier itself. This research incorporates a different statistic that estimates the BER, or the probability of misclassification given some data, to serve as an upper bound for potential classifier performance. This estimate leverages a Friedman-Rafsky test statistic: the number of cross labels in a minimum-spanning tree (MST) through points in the feature space. The first part of this study examines the behavior of the BER estimate over a general learning process, such as different epochs of the training process in a neural network. The second part of the study examines whether certain factors affect the separability of synthetic aperture radar (SAR) images of targets of interest. Given the fact that it is often difficult and expensive to survey real targets and generate SAR images, 3-D CAD models are frequently used to generate synthetic SAR images. However, given that many resources are devoted to perfecting these models, this study applies the BER estimate to examine whether minute changes to CAD models affect separability in the image domain. The results seem to indicate that, if the topology of the target is maintained in the CAD domain, low-fidelity versions of targets (with 25% the number of faces of highly accurate models), exhibit separability and ability to be correctly classified identical to their high-fidelity counterparts. The BER estimation also shows promising applications in other domains, serving as a way to describe the underlying structure of feature spaces intuitively but effectively.
|
