Please use this identifier to cite or link to this item: http://hdl.handle.net/10397/18356
Title: Measuring relevance between discrete and continuous features based on neighborhood mutual information
Authors: Hu, Q
Zhang, L 
Zhang, D 
Pan, W
An, S
Pedrycz, W
Keywords: Continuous feature
Feature selection
Neighborhood entropy
Neighborhood mutual information
Relevance
Issue Date: 2011
Publisher: Pergamon Press
Source: Expert systems with applications, 2011, v. 38, no. 9, p. 10737-10750 How to cite?
Journal: Expert systems with applications 
Abstract: Measures of relevance between features play an important role in classification and regression analysis. Mutual information has been proved an effective measure for decision tree construction and feature selection. However, there is a limitation in computing relevance between numerical features with mutual information due to problems of estimating probability density functions in high-dimensional spaces. In this work, we generalize Shannon's information entropy to neighborhood information entropy and propose a measure of neighborhood mutual information. It is shown that the new measure is a natural extension of classical mutual information which reduces to the classical one if features are discrete; thus the new measure can also be used to compute the relevance between discrete variables. In addition, the new measure introduces a parameter delta to control the granularity in analyzing data. With numeric experiments, we show that neighborhood mutual information produces the nearly same outputs as mutual information. However, unlike mutual information, no discretization is required in computing relevance when used the proposed algorithm. We combine the proposed measure with four classes of evaluating strategies used for feature selection. Finally, the proposed algorithms are tested on several benchmark data sets. The results show that neighborhood mutual information based algorithms yield better performance than some classical ones.
URI: http://hdl.handle.net/10397/18356
ISSN: 0957-4174
EISSN: 1873-6793
DOI: 10.1016/j.eswa.2011.01.023
Appears in Collections:Journal/Magazine Article

Access
View full-text via PolyU eLinks SFX Query
Show full item record

SCOPUSTM   
Citations

40
Last Week
0
Last month
0
Citations as of Oct 10, 2017

WEB OF SCIENCETM
Citations

28
Last Week
0
Last month
3
Citations as of Oct 23, 2017

Page view(s)

57
Last Week
1
Last month
Checked on Oct 22, 2017

Google ScholarTM

Check

Altmetric



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.