Some relationships between minimum Bayes error and information theoretical feature extraction

Manuela Vasconcelos; Nuno Vasconcelos

doi:10.1117/12.604289

19 May 2005 Some relationships between minimum Bayes error and information theoretical feature extraction

Manuela Vasconcelos, Nuno Vasconcelos

Proceedings Volume 5807, Automatic Target Recognition XV; (2005) https://doi.org/10.1117/12.604289
Event: Defense and Security, 2005, Orlando, Florida, United States

Abstract

Feature extraction and selection are important problems in statistical learning. We study the relationships between two previously proposed principles for their optimal solution: the minimization of Bayes error and the maximization of mutual information between features and class labels. It is shown that a quantity which provides insight on this relationship is the set of non-increasing probability mass functions (NIPMFs). We derive some basic properties of the members of this set, show that any classification problem defines an ensemble of NIPMFs, and that the probability distribution of this ensemble uniquely determines the associated Bayes error and mutual information. These results are then used to show that, when the classification problem is binary and some generic constraints hold, the optimal feature space is the same under the two formulations.

Citation Download Citation

Manuela Vasconcelos and Nuno Vasconcelos "Some relationships between minimum Bayes error and information theoretical feature extraction", Proc. SPIE 5807, Automatic Target Recognition XV, (19 May 2005); https://doi.org/10.1117/12.604289

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available