|
|
| (379 intermediate revisions by more than 100 users not shown) |
| Line 1: |
Line 1: |
| {{machine learning bar}}
| | This is a preview for the new '''MathML rendering mode''' (with SVG fallback), which is availble in production for registered users. |
| In [[machine learning]] and [[statistics]], '''feature selection''', also known as '''variable selection''', '''attribute selection''' or '''variable subset selection''', is the process of selecting a subset of relevant features for use in model construction.
| |
| The central assumption when using a feature selection technique is that the data contains many ''redundant'' or ''irrelevant'' features. Redundant features are those which provide no more information than the currently selected features, and irrelevant features provide no useful information in any context.
| |
| Feature selection techniques are a subset of the more general field of [[feature extraction]]. Feature extraction creates new features from functions of the original features, whereas feature selection returns a subset of the features.
| |
| Feature selection techniques are often used in domains where there are many features and comparatively few samples (or data points). The archetypal case is the use of feature selection in analysing [[DNA microarray]]s, where there are many thousands of features, and a few tens to hundreds of samples. Feature selection techniques provide three main benefits when constructing predictive models:
| |
| :* improved model interpretability,
| |
| :* shorter training times,
| |
| :* enhanced generalisation by reducing [[overfitting]].
| |
| Feature selection is also useful as part of the data analysis process, as it shows which features are important for prediction, and how these features are related.
| |
|
| |
|
| ==Introduction==
| | If you would like use the '''MathML''' rendering mode, you need a wikipedia user account that can be registered here [[https://en.wikipedia.org/wiki/Special:UserLogin/signup]] |
| A feature selection algorithm can be seen as the combination of a search technique for proposing new feature subsets, along with an evaluation measure which scores the different feature subsets.
| | * Only registered users will be able to execute this rendering mode. |
| The simplest algorithm is to test each possible subset of features finding the one which minimises the error rate. This is an exhaustive search of the space, and is computationally intractable for all but the smallest of feature sets.
| | * Note: you need not enter a email address (nor any other private information). Please do not use a password that you use elsewhere. |
| The choice of evaluation metric heavily influences the algorithm, and it is these evaluation metrics which distinguish between the three main categories of feature selection algorithms: wrappers, filters and embedded methods.<ref>http://jmlr.csail.mit.edu/papers/v3/guyon03a.html</ref>
| |
|
| |
|
| Wrapper methods use a predictive model to score feature subsets. Each new subset is used to train a model, which is tested on a hold-out set. Counting the number of mistakes made on that hold-out set (the error rate of the model) gives the score for that subset. As wrapper methods train a new model for each subset, they are very computationally intensive, but usually provide the best performing feature set for that particular type of model.
| | Registered users will be able to choose between the following three rendering modes: |
|
| |
|
| Filter methods use a proxy measure instead of the error rate to score a feature subset. This measure is chosen to be fast to compute, whilst still capturing the usefulness of the feature set. Common measures include the [[pointwise mutual information]],<ref name="textcat"/> [[Pearson product-moment correlation coefficient]], inter/intra class distance or the scores of [[Statistical hypothesis testing|significance tests]] for each class/feature combinations.<ref name="textcat">{{cite conference |last1=Yang |first1=Yiming |first2=Jan O. |last2=Pedersen |title=A comparative study on feature selection in text categorization |conference=ICML |year=1997}}</ref><ref>{{cite journal |last1=Forman |first1=George |title=An extensive empirical study of feature selection metrics for text classification |journal=Journal of Machine Learning Research |volume=3 |year=2003 |pages=1289–1305}}</ref>
| | '''MathML''' |
| Filters are usually less computationally intensive than wrappers, but they produce a feature set which is not tuned to a specific type of predictive model. Many filters provide a feature ranking rather than an explicit best feature subset, and the cut off point in the ranking is chosen via [[Cross-validation (statistics)|cross-validation]].
| | :<math forcemathmode="mathml">E=mc^2</math> |
|
| |
|
| Embedded methods are a catch-all group of techniques which perform feature selection as part of the model construction process. The exemplar of this approach is the [[Least squares#Regularized versions|LASSO]] method for constructing a linear model, which penalises the regression coefficients, shrinking many of them to zero. Any features which have non-zero regression coefficients are 'selected' by the LASSO algorithm. One other popular approach is the Recursive Feature Elimination algorithm, commonly used with [[Support Vector Machines]] to repeatedly construct a model and remove features with low weights. These approaches tend to be between filters and wrappers in terms of computational complexity.
| | <!--'''PNG''' (currently default in production) |
| | :<math forcemathmode="png">E=mc^2</math> |
|
| |
|
| In statistics, the most popular form of feature selection is [[stepwise regression]]. It is a [[greedy algorithm]] that adds the best feature (or deletes the worst feature) at each round. The main control issue is deciding when to stop the algorithm. In machine learning, this is typically done by [[Cross-validation (statistics)|cross-validation]]. In statistics, some criteria are optimized. This leads to the inherent problem of nesting. More robust methods have been explored, such as [[branch and bound]] and piecewise linear network.
| | '''source''' |
| | :<math forcemathmode="source">E=mc^2</math> --> |
|
| |
|
| ==Subset selection== | | <span style="color: red">Follow this [https://en.wikipedia.org/wiki/Special:Preferences#mw-prefsection-rendering link] to change your Math rendering settings.</span> You can also add a [https://en.wikipedia.org/wiki/Special:Preferences#mw-prefsection-rendering-skin Custom CSS] to force the MathML/SVG rendering or select different font families. See [https://www.mediawiki.org/wiki/Extension:Math#CSS_for_the_MathML_with_SVG_fallback_mode these examples]. |
| Subset selection evaluates a subset of features as a group for suitability. Subset selection algorithms can be broken up into Wrappers, Filters and Embedded. Wrappers use a search algorithm to search through the space of possible features and evaluate each subset by running a model on the subset. Wrappers can be computationally expensive and have a risk of over fitting to the model. Filters are similar to Wrappers in the search approach, but instead of evaluating against a model, a simpler filter is evaluated. Embedded techniques are embedded in and specific to a model.
| |
|
| |
|
| Many popular search approaches use [[greedy algorithm|greedy]] [[hill climbing]], which iteratively evaluates a candidate subset of features, then modifies the subset and evaluates if the new subset is an improvement over the old. Evaluation of the subsets requires a scoring [[Metric (mathematics)|metric]] that grades a subset of features. Exhaustive search is generally impractical, so at some implementor (or operator) defined stopping point, the subset of features with the highest score discovered up to that point is selected as the satisfactory feature subset. The stopping criterion varies by algorithm; possible criteria include: a subset score exceeds a threshold, a program's maximum allowed run time has been surpassed, etc.
| | ==Demos== |
|
| |
|
| Alternative search-based techniques are based on [[targeted projection pursuit]] which finds low-dimensional projections of the data that score highly: the features that have the largest projections in the lower dimensional space are then selected.
| | Here are some [https://commons.wikimedia.org/w/index.php?title=Special:ListFiles/Frederic.wang demos]: |
|
| |
|
| Search approaches include:
| |
|
| |
|
| * Exhaustive | | * accessibility: |
| * Best first | | ** Safari + VoiceOver: [https://commons.wikimedia.org/wiki/File:VoiceOver-Mac-Safari.ogv video only], [[File:Voiceover-mathml-example-1.wav|thumb|Voiceover-mathml-example-1]], [[File:Voiceover-mathml-example-2.wav|thumb|Voiceover-mathml-example-2]], [[File:Voiceover-mathml-example-3.wav|thumb|Voiceover-mathml-example-3]], [[File:Voiceover-mathml-example-4.wav|thumb|Voiceover-mathml-example-4]], [[File:Voiceover-mathml-example-5.wav|thumb|Voiceover-mathml-example-5]], [[File:Voiceover-mathml-example-6.wav|thumb|Voiceover-mathml-example-6]], [[File:Voiceover-mathml-example-7.wav|thumb|Voiceover-mathml-example-7]] |
| * [[Simulated annealing]] | | ** [https://commons.wikimedia.org/wiki/File:MathPlayer-Audio-Windows7-InternetExplorer.ogg Internet Explorer + MathPlayer (audio)] |
| * [[Genetic algorithm]]
| | ** [https://commons.wikimedia.org/wiki/File:MathPlayer-SynchronizedHighlighting-WIndows7-InternetExplorer.png Internet Explorer + MathPlayer (synchronized highlighting)] |
| * [[Greedy algorithm|Greedy]] forward selection
| | ** [https://commons.wikimedia.org/wiki/File:MathPlayer-Braille-Windows7-InternetExplorer.png Internet Explorer + MathPlayer (braille)] |
| * [[Greedy algorithm|Greedy]] backward elimination
| | ** NVDA+MathPlayer: [[File:Nvda-mathml-example-1.wav|thumb|Nvda-mathml-example-1]], [[File:Nvda-mathml-example-2.wav|thumb|Nvda-mathml-example-2]], [[File:Nvda-mathml-example-3.wav|thumb|Nvda-mathml-example-3]], [[File:Nvda-mathml-example-4.wav|thumb|Nvda-mathml-example-4]], [[File:Nvda-mathml-example-5.wav|thumb|Nvda-mathml-example-5]], [[File:Nvda-mathml-example-6.wav|thumb|Nvda-mathml-example-6]], [[File:Nvda-mathml-example-7.wav|thumb|Nvda-mathml-example-7]]. |
| * [[Targeted projection pursuit]]
| | ** Orca: There is ongoing work, but no support at all at the moment [[File:Orca-mathml-example-1.wav|thumb|Orca-mathml-example-1]], [[File:Orca-mathml-example-2.wav|thumb|Orca-mathml-example-2]], [[File:Orca-mathml-example-3.wav|thumb|Orca-mathml-example-3]], [[File:Orca-mathml-example-4.wav|thumb|Orca-mathml-example-4]], [[File:Orca-mathml-example-5.wav|thumb|Orca-mathml-example-5]], [[File:Orca-mathml-example-6.wav|thumb|Orca-mathml-example-6]], [[File:Orca-mathml-example-7.wav|thumb|Orca-mathml-example-7]]. |
| * Scatter Search<ref>F.C. Garcia-Lopez, M. Garcia-Torres, B. Melian, J.A. Moreno-Perez, J.M. Moreno-Vega. Solving feature subset selection problem by a Parallel Scatter Search, ''European Journal of Operational Research'', vol. 169, no. 2, pp. 477–489, 2006. | | ** From our testing, ChromeVox and JAWS are not able to read the formulas generated by the MathML mode. |
| </ref>
| |
| * Variable Neighborhood Search<ref>F.C. Garcia-Lopez, M. Garcia-Torres, B. Melian, J.A. Moreno-Perez, J.M. Moreno-Vega. Solving Feature Subset Selection Problem by a Hybrid Metaheuristic. In ''First International Workshop on Hybrid Metaheuristics'', pp. 59–68, 2004.</ref> | |
|
| |
|
| Two popular filter metrics for classification problems are [[correlation]] and [[mutual information]], although neither are true [[metric (mathematics)|metrics]] or 'distance measures' in the mathematical sense, since they fail to obey the [[triangle inequality]] and thus do not compute any actual 'distance' – they should rather be regarded as 'scores'. These scores are computed between a candidate feature (or set of features) and the desired output category. There are, however, true metrics that are a simple function of the mutual information;<ref>Alexander Kraskov, Harald Stögbauer, Ralph G. Andrzejak, and [[Peter Grassberger]], "Hierarchical Clustering Based on Mutual Information", (2003) ''[http://arxiv.org/abs/q-bio/0311039 ArXiv q-bio/0311039]''</ref> see [[mutual information#Metric|here]].
| | ==Test pages == |
|
| |
|
| Other available filter metrics include:
| | To test the '''MathML''', '''PNG''', and '''source''' rendering modes, please go to one of the following test pages: |
| | *[[Displaystyle]] |
| | *[[MathAxisAlignment]] |
| | *[[Styling]] |
| | *[[Linebreaking]] |
| | *[[Unique Ids]] |
| | *[[Help:Formula]] |
|
| |
|
| * Class separability
| | *[[Inputtypes|Inputtypes (private Wikis only)]] |
| ** Error probability
| | *[[Url2Image|Url2Image (private Wikis only)]] |
| ** Inter-class distance
| | ==Bug reporting== |
| ** Probabilistic distance
| | If you find any bugs, please report them at [https://bugzilla.wikimedia.org/enter_bug.cgi?product=MediaWiki%20extensions&component=Math&version=master&short_desc=Math-preview%20rendering%20problem Bugzilla], or write an email to math_bugs (at) ckurs (dot) de . |
| ** [[Entropy (Information theory)|Entropy]]
| |
| * Consistency-based feature selection | |
| * Correlation-based feature selection
| |
| | |
| ==Optimality criteria==
| |
| | |
| There are a variety of optimality criteria that can be used for controlling feature selection. The oldest are [[Mallows's Cp|Mallows's ''C<sub>p</sub>'']] statistic and [[Akaike information criterion]] (AIC). These add variables if the [[Student's t-test|''t''-statistic]] is bigger than <math>\sqrt{2}</math>.
| |
| | |
| Other criteria are [[Bayesian information criterion]] (BIC) which uses <math>\sqrt{\log{n}}</math>, [[minimum description length]] (MDL) which asymptotically uses <math>\sqrt{\log{n}}</math>, Bonnferroni / [[Risk Inflation Criterion|RIC]] which use <math>\sqrt{2\log{p}}</math>, maximum dependency feature selection, and a variety of new criteria that are motivated by [[false discovery rate]] (FDR) which use something close to <math>\sqrt{2\log{\frac{p}{q}}}</math>.
| |
| | |
| ==Minimum-redundancy-maximum-relevance (mRMR) feature selection== | |
| Peng ''et al.''<ref>{{cite journal |last1=Peng |first1=H. C. |last2=Long |first2=F. |last3=Ding |first3=C. |title=Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy |journal=IEEE Transactions on Pattern Analysis and Machine Intelligence |volume=27 |issue=8 |pages=1226–1238 |year=2005 |doi=10.1109/TPAMI.2005.159 |pmid=16119262}} [http://penglab.janelia.org/proj/mRMR/index.htm Program]</ref> proposed an mRMR feature-selection method that can use either mutual information, correlation, distance/similarity scores to select features. For example, with mutual information, relevant features and redundant features are considered simultaneously. The relevance of a feature set ''S'' for the class ''c'' is defined by the average value of all mutual information values between the individual feature ''f<sub>i</sub>'' and the class ''c'' as follows:
| |
| | |
| <math> D(S,c) = \frac{1}{|S|}\sum_{f_{i}\in S}I(f_{i};c) </math>.
| |
| | |
| The redundancy of all features in the set ''S'' is the average value of all mutual information values between the feature ''f<sub>i</sub>'' and the feature ''f<sub>j</sub>'':
| |
| | |
| <math> R(S) = \frac{1}{|S|^{2}}\sum_{f_{i},f_{j}\in S}I(f_{i};f_{j})</math>
| |
| | |
| The mRMR criterion is a combination of two measures given above and is defined as follows:
| |
| | |
| <math>\mathrm{mRMR}= \max_{S}
| |
| \left[\frac{1}{|S|}\sum_{f_{i}\in S}I(f_{i};c) -
| |
| \frac{1}{|S|^{2}}\sum_{f_{i},f_{j}\in S}I(f_{i};f_{j})\right].</math>
| |
| | |
| Suppose that there are ''n'' full-set features. Let ''x<sub>i</sub>'' be the set membership [[indicator function]] for feature ''f<sub>i</sub>'', so that ''x<sub>i</sub>''=1 indicates presence and ''x<sub>i</sub>''=0 indicates absence of the feature ''f<sub>i</sub>'' in the globally optimal feature set. Let ''c<sub>i</sub>=I(f<sub>i</sub>;c)'' and ''a<sub>ij</sub>=I(f<sub>i</sub>;f<sub>j</sub>)''. The above may then be written as an optimization problem:
| |
| | |
| <math>\mathrm{mRMR}= \max_{x\in \{0,1\}^{n}}
| |
| \left[\frac{\sum^{n}_{i=1}c_{i}x_{i}}{\sum^{n}_{i=1}x_{i}} -
| |
| \frac{\sum^{n}_{i,j=1}a_{ij}x_{i}x_{j}}
| |
| {(\sum^{n}_{i=1}x_{i})^{2}}\right].</math>
| |
| | |
| It may be shown that mRMR feature selection is an approximation of the theoretically optimal maximum-dependency feature selection that maximizes the mutual information between the joint distribution of the selected features and the classification variable. However, since mRMR turned a combinatorial problem as a series of much smaller scale problems, each of which only involves two variables, the estimation of joint probabilities is much more robust. In certain situations the algorithm may underestimate the usefulness of features as it has no way to measure interactions between features. This can lead to poor performance<ref>Brown, G., Pocock, A., Zhao, M.-J., Lujan, M. (2012). "Conditional Likelihood Maximisation: A Unifying Framework for Information Theoretic Feature Selection", In the Journal of Machine Learning Research (JMLR). [http://www.jmlr.org/papers/volume13/brown12a/brown12a.pdf]</ref> when the features are individually useless, but are useful when combined (a pathological case is found when the class is a [[parity function]] of the features). Overall the algorithm is more efficient (in terms of the amount of data required) than the theoretically optimal max-dependency selection, yet produces a low redundancy feature set.
| |
| | |
| The mRMR method has also been combined with the wrapper methods, thus a wrapper method can be utilized at a smaller cost. It can be seen that mRMR is also related to the correlation based feature selection below. It may also be seen a special case of some generic feature selectors.<ref name="docs.google">Nguyen, H., Franke, K., Petrovic, S. (2010). "Towards a Generic Feature-Selection Measure for Intrusion Detection", In Proc. International Conference on Pattern Recognition (ICPR), Istanbul, Turkey. [https://www.researchgate.net/publication/220928649_Towards_a_Generic_Feature-Selection_Measure_for_Intrusion_Detection?ev=prf_pub]</ref>
| |
| | |
| ==Correlation feature selection==
| |
| The Correlation Feature Selection (CFS) measure evaluates subsets of features on the basis of the following hypothesis: "Good feature subsets contain features highly correlated with the classification, yet uncorrelated to each other".<ref>M. Hall 1999, [http://www.cs.waikato.ac.nz/~mhall/thesis.pdf Correlation-based Feature Selection for Machine Learning]</ref> <ref>Senliol, Baris, et al. "Fast Correlation Based Filter (FCBF) with a different search strategy." Computer and Information Sciences, 2008. ISCIS'08. 23rd International Symposium on. IEEE, 2008. [http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=4717949]</ref> The following equation gives the merit of a feature subset ''S'' consisting of ''k'' features:
| |
| | |
| <math> Merit_{S_{k}} = \frac{k\overline{r_{cf}}}{\sqrt{k+k(k-1)\overline{r_{ff}}}}.</math>
| |
|
| |
| Here, <math> \overline{r_{cf}} </math> is the average value of all feature-classification correlations, and <math> \overline{r_{ff}} </math> is the average value of all feature-feature correlations. The CFS criterion is defined as follows:
| |
| | |
| <math>\mathrm{CFS} = \max_{S_k}
| |
| \left[\frac{r_{c f_1}+r_{c f_2}+\cdots+r_{c f_k}}
| |
| {\sqrt{k+2(r_{f_1 f_2}+\cdots+r_{f_i f_j}+ \cdots
| |
| + r_{f_k f_1 })}}\right].</math>
| |
| | |
| The <math>r_{cf_{i}}</math> and <math>r_{f_{i}f_{j}}</math> variables are referred to as correlations, but are not necessarily [[Pearson product-moment correlation coefficient|Pearson's correlation coefficient]] or [[Spearman's rank correlation coefficient|Spearman's ρ]]. Dr. Mark Hall's dissertation uses neither of these, but uses three different measures of relatedness, [[minimum description length]] (MDL), [[Mutual Information#Normalized variants|symmetrical uncertainty]], and [[Relief (feature selection)|relief]].
| |
| | |
| Let ''x<sub>i</sub>'' be the set membership [[indicator function]] for feature ''f<sub>i</sub>''; then the above can be rewritten as an optimization problem:
| |
| | |
| <math>\mathrm{CFS} = \max_{x\in \{0,1\}^{n}}
| |
| \left[\frac{(\sum^{n}_{i=1}a_{i}x_{i})^{2}}
| |
| {\sum^{n}_{i=1}x_i + \sum_{i\neq j} 2b_{ij} x_i x_j }\right].</math>
| |
| | |
| The combinatorial problems above are, in fact, mixed 0–1 [[linear programming]] problems that can be solved by using [[branch-and-bound algorithm]]s.<ref>Hai Nguyen, Katrin Franke, and Slobodan Petrovic, Optimizing a class of feature selection measures, Proceedings of the NIPS 2009 Workshop on Discrete Optimization in Machine Learning: Submodularity, Sparsity & Polyhedra (DISCML), Vancouver, Canada, December 2009. [https://www.researchgate.net/publication/231175763_Optimizing_a_Class_of_Feature_Selection_Measures?ev=prf_pub]</ref>
| |
| | |
| ==Regularized trees==
| |
| The features from a decision tree or a tree ensemble are shown to be redundant. A recent method called regularized tree<ref>H. Deng, G. Runger, "[https://sites.google.com/site/houtaodeng/publications/FSRegularizedTrees.pdf?attredirects=0 Feature Selection via Regularized Trees]", Proceedings of the 2012 International Joint Conference on Neural Networks (IJCNN), IEEE, 2012</ref> can be used for feature subset selection. Regularized trees penalize using a variable similar to the variables selected at previous tree nodes for splitting the current node. Regularized trees only need build one tree model (or one tree ensemble model) and thus are computationally efficient.
| |
| | |
| Regularized trees naturally handle numerical and categorical features, interactions and nonlinearities. They are invariant to attribute scales (units) and insensitive to outliers, and thus, require little data preprocessing such as normalization. Regularized random forest (RRF) ([http://cran.r-project.org/web/packages/RRF/index.html RRF]) is one type of regularized trees. The guided RRF is an enhanced RRF which is guided by the importance scores from an ordinary random forest.
| |
| | |
| ==Embedded methods incorporating feature selection==
| |
| * [[Random multinomial logit]] (RMNL)
| |
| * Sparse regression, LASSO
| |
| * [http://arxiv.org/abs/1201.1587 Regularized trees] e.g. regularized random forest implemented in the [http://cran.r-project.org/web/packages/RRF/index.html RRF] package
| |
| * [[Decision tree learning|Decision tree]]
| |
| * [[Memetic algorithm]]
| |
| * Auto-encoding networks with a bottleneck-layer
| |
| * Many other [[machine learning]] methods applying a [[Pruning (algorithm)|pruning]] step.
| |
| | |
| ==Software for feature selection==
| |
| | |
| Many standard [[:Category:Data analysis software|data analysis software systems]] are often used for feature selection, such as [[SciLab]], [[NumPy]] and [[R (programming language)|the R language]]. Other software systems are tailored specifically to the feature-selection task:
| |
| | |
| * [[Weka (machine learning)|Weka]] – freely available and [[open source|open-source]] software in Java.
| |
| * [[Feature Selection Toolbox|Feature Selection Toolbox 3]] – freely available and [[open source|open-source]] software in C++.
| |
| * [[RapidMiner]] – freely available and [[open source|open-source]] software.
| |
| * [[Orange (software)|Orange]] – freely available and [[open source|open-source]] software (module [http://www.ailab.si/orange/doc/modules/orngFSS.htm orngFSS]).
| |
| * [http://sites.google.com/site/tooldiag/ TOOLDIAG Pattern recognition toolbox] – freely available C toolbox.
| |
| * [http://penglab.janelia.org/proj/mRMR/ minimum redundancy feature selection tool] – freely available C/Matlab codes for selecting minimum redundant features.
| |
| * [http://web.archive.org/web/20110718043215/http://links.cse.msu.edu:8000/members/matt_gerber/index.php/Machine_learning_software A C# Implementation] of greedy forward feature subset selection for various classifiers (e.g., LibLinear, SVM-light).
| |
| * [http://www.ipipan.eu/staff/m.draminski/files/dmLab170.zip MCFS-ID] (Monte Carlo Feature Selection and Interdependency Discovery) is a Monte Carlo method-based tool for feature selection. It also allows for the discovery of interdependencies between the relevant features. MCFS-ID is particularly suitable for the analysis of high-dimensional, ill-defined transactional and biological data.
| |
| * [http://cran.r-project.org/web/packages/RRF/index.html RRF] is an R package for feature selection and can be installed from R. RRF stands for Regularized Random Forest, which is a type of Regularized Trees. By building a regularized random forest, a compact set of non-redundant features can be selected without loss of predictive information. Regularized trees can capture non-linear interactions between variables, and naturally handle different scales, and numerical and categorical variables.
| |
| | |
| ==See also==
| |
| * [[Cluster analysis]]
| |
| * [[Dimensionality reduction]]
| |
| * [[Feature extraction]]
| |
| * [[Data mining]]
| |
| | |
| {{More footnotes|date=July 2010}}
| |
| | |
| ==References==
| |
| | |
| {{Reflist|30em}}
| |
| | |
| ==Further reading==
| |
| | |
| {{Refbegin}}
| |
| * [http://featureselection.asu.edu/featureselection_techreport.pdf Tutorial Outlining Feature Selection Algorithms, Arizona State University]
| |
| * [http://jmlr.csail.mit.edu/papers/special/feature03.html JMLR Special Issue on Variable and Feature Selection]
| |
| * [http://www.springer.com/west/home?SGWID=4-102-22-33327495-0&changeHeader=true&referer=www.wkap.nl&SHORTCUT=www.springer.com/prod/b/0-7923-8198-X Feature Selection for Knowledge Discovery and Data Mining] (Book)
| |
| * [http://jmlr.csail.mit.edu/papers/volume3/guyon03a/guyon03a.pdf An Introduction to Variable and Feature Selection] (Survey)
| |
| * [http://ieeexplore.ieee.org/iel5/69/30435/01401889.pdf Toward integrating feature selection algorithms for classification and clustering] (Survey)
| |
| * [http://library.utia.cas.cz/separaty/2010/RO/somol-efficient%20feature%20subset%20selection%20and%20subset%20size%20optimization.pdf Efficient Feature Subset Selection and Subset Size Optimization] (Survey, 2010)
| |
| * [http://www.ijcai.org/papers07/Papers/IJCAI07-187.pdf Searching for Interacting Features]
| |
| * [http://www.autonlab.org/icml_documents/camera-ready/107_Feature_Subset_Selec.pdf Feature Subset Selection Bias for Classification Learning]
| |
| * Y. Sun, S. Todorovic, S. Goodison, [http://plaza.ufl.edu/sunyijun/PAMI2.htm Local Learning Based Feature Selection for High-dimensional Data Analysis], ''IEEE Transactions on Pattern Analysis and Machine Intelligence'', vol. 32, no. 9, pp. 1610–1626, 2010.
| |
| {{Refend}}
| |
| | |
| ==External links==
| |
| * [http://featureselection.asu.edu/software.php Feature Selection Package, Arizona State University (Matlab Code)]
| |
| * [http://www.clopinet.com/isabelle/Projects/NIPS2003/ NIPS challenge 2003] (see also [[NIPS]])
| |
| * [http://paul.luminos.nl/documents/show_document.php?d=198 Naive Bayes implementation with feature selection in Visual Basic] (includes executable and source code)
| |
| * [http://penglab.janelia.org/proj/mRMR/index.htm Minimum-redundancy-maximum-relevance (mRMR) feature selection program]
| |
| * [http://mloss.org/software/view/386/ FEAST] (Open source Feature Selection algorithms in C and MATLAB)
| |
| | |
| [[Category:Model selection]]
| |
| [[Category:Dimension reduction]]
| |
