Multimodal data fusion using signal/image processing methods for multi-class machine learning

Casey J. Richards; Nawal Valliani; Benjamin A. Johnson; Nelson Ka Ki Wong; Angelo Pennati; Amir K. Saeed; Benjamin M. Rodriguez

doi:10.1117/12.2664987

14 June 2023 Multimodal data fusion using signal/image processing methods for multi-class machine learning

Casey J. Richards, Nawal Valliani, Benjamin A. Johnson, Nelson Ka Ki Wong, Angelo Pennati, Amir K. Saeed, Benjamin M. Rodriguez

Author Affiliations +

Proceedings Volume 12547, Signal Processing, Sensor/Information Fusion, and Target Recognition XXXII; 125470N (2023) https://doi.org/10.1117/12.2664987
Event: SPIE Defense + Commercial Sensing, 2023, Orlando, Florida, United States

Abstract

As the world progresses further into the digital era, we see a growing utility for combining datasets gathered on different devices and receivers as well as on varying time ranges, for use in machine learning. However, machine learning classification introduces a requirement for standardized data, which in turn hampers the ability to utilize diverse sets of data at a given timestamp. In this paper, we investigate the application of various signal pre-processing techniques (Daubecheis wavelet, discrete cosine and discrete fourier transform among others) for multi-modal, multi-class machine learning. Following the pre-processing, the multi-faceted signals are represented solely by features generated from first order statistics, eigen decomposition, and linear discriminant. Utilizing these generated features, as opposed to the signals themselves, these diverse datasets may now be combined as input to machine learning methods. Furthermore, we apply Fisher’s linear discriminant ratio and Random Forest feature importance metrics for feature ranking and feature space reduction followed by a comparison of the approaches. Our work demonstrates that dissimilar datasets with common classes may be combined using the proposed methods with a classification accuracy ≥ 95%. This paper demonstrates that the feature space may be reduced by approximately 60% with ≤ 5% loss in classification accuracy, and in some cases, a slight increase in classification accuracy.

Conference Presentation

Citation Download Citation

Casey J. Richards, Nawal Valliani, Benjamin A. Johnson, Nelson Ka Ki Wong, Angelo Pennati, Amir K. Saeed, and Benjamin M. Rodriguez "Multimodal data fusion using signal/image processing methods for multi-class machine learning", Proc. SPIE 12547, Signal Processing, Sensor/Information Fusion, and Target Recognition XXXII, 125470N (14 June 2023); https://doi.org/10.1117/12.2664987

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available