|
|
| Line 1: |
Line 1: |
| In [[statistics]] and, in particular, in the fitting of [[linear regression|linear]] or [[logistic regression]] models, the '''elastic net''' is a [[Regularization (mathematics)|regularized]] regression method that [[Linear combination|linearly combines]] the [[Taxicab geometry|L1]] and [[Norm (mathematics)#p-norm|L2]] penalties of the [[ Least_squares#Lasso_method| lasso]] and [[Tikhonov regularization| ridge]] methods.
| | ICT Sales Representative Sirmans from Melville, has hobbies and interests which includes comics, airsoft and string figures. Will soon undertake a contiki journey which will include touring the Pirin National Park.<br><br>Here is my blog; [http://comoficarrico.comoganhardinheiro101.com/ como ganhar dinheiro na internet] |
| | |
| ==Specification==
| |
| | |
| The elastic net method overcomes the limitations of the [[ Least_squares#Lasso_method| LASSO (least absolute shrinkage and selection operator)]] method which uses a penalty function based on
| |
| :<math>\|\beta\|_1 = \textstyle \sum_{j=1}^p |\beta_j|.</math>
| |
| Use of this penalty function has several limitations.<ref name=ZH>{{cite journal|last1=Zou|first1=Hui|first2=Trevor|last2=Hastie|date=2005|title=Regularization and Variable Selection via the Elastic Net|journal=[[Journal of the Royal Statistical Society]], Series B|pages=301–320|url=http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.124.4696}}</ref> For example, in the "large ''p'', small ''n''" case (high dimensional data with few examples), the LASSO selects at most n variables before it saturates. Also if there is a group of highly correlated variables, then the LASSO tends to select one variable from a group and ignore the others. To overcome these limitations, the elastic net adds a quadratic part to the penalty (<math>\|\beta\|^2</math>), which when used alone is [[ridge regression]] (known also as [[Tikhonov regularization]]). The estimates from the elastic net method are defined by
| |
| | |
| : <math> \hat{\beta} = \underset{\beta}{\operatorname{argmin}} (\| y-X \beta \|^2 + \lambda_2 \|\beta\|^2 + \lambda_1 \|\beta\|_1) .</math>
| |
| | |
| As a result, the elastic net method includes the LASSO and ridge regression: in other words, each of them is a special case where <math>\lambda_1 = \lambda, \lambda_2 = 0</math> or <math>\lambda_1 = 0, \lambda_2 = \lambda</math>. Meanwhile, the naive version of elastic net method finds an estimator in a two-stage procedure : first for each fixed <math>\lambda_2</math> it finds the ridge regression coefficients, and then does a LASSO type shrinkage. This kind of estimation incurs a double amount of shrinkage, which introduces unnecessary extra bias and outcomes with bad prediction performance. To improve the prediction performance, the authors rescale the coefficients of the naive version of elastic net by multiplying the estimated coefficients by <math>(1 + \lambda_2)</math>.<ref name=ZH/>
| |
| | |
| == Software ==
| |
| * "Glmnet: Lasso and elastic-net regularized generalized linear models" is software which is implemented as an [[R (programming language)|R]] source package.<ref>{{cite journal|last=Friedman|first=Jerome|coauthors=Trevor Hastie and Rob Tibshirani|date=2010|title=Regularization Paths for Generalized Linear Models via Coordinate Descent|journal=Journal of Statistical Software|pages=1–22|url=http://www.jstatsoft.org/v33/i01}}</ref><ref>http://cran.r-project.org/web/packages/glmnet/index.html</ref> This includes fast algorithms for estimation of generalized linear models with ℓ<sub>1</sub> (the lasso), ℓ<sub>2</sub> (ridge regression) and mixtures of the two penalties (the elastic net) using cyclical coordinate descent, computed along a regularization path.
| |
| *[[JMP (statistical software)|JMP Pro 11]] includes elastic net regularization, using the Generalized Regression personality with Fit Model.
| |
| * "pensim: Simulation of high-dimensional data and parallelized repeated penalized regression" implements an alternate, parallelised "2D" tuning method of the ℓ parameters, a method claimed to result in improved prediction accuracy.<ref>{{Cite pmid|22156367}}</ref><ref>http://cran.r-project.org/web/packages/pensim/index.html</ref>
| |
| * [[scikit-learn]] includes linear regression, [[logistic regression]] and linear [[support vector machine]]s with elastic net regularization.
| |
| | |
| == References ==
| |
| {{Reflist}}
| |
| | |
| == External links ==
| |
| * [http://www.stanford.edu/~hastie/TALKS/enet_talk.pdf http://www.stanford.edu/~hastie/TALKS/enet_talk.pdf]
| |
| | |
| <!--- Categories --->
| |
| [[Category:Regression analysis]]
| |
ICT Sales Representative Sirmans from Melville, has hobbies and interests which includes comics, airsoft and string figures. Will soon undertake a contiki journey which will include touring the Pirin National Park.
Here is my blog; como ganhar dinheiro na internet