Volume 8, Issue 2 (Vol. 8,No. 2, 2022)
mmr 2022, 8(2): 152-164 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Alimohammadi R, Bahari J. a simulation comparison of Ridge regression estimators with Lars. mmr 2022; 8 (2) :152-164
URL: http://mmr.khu.ac.ir/article-1-2942-en.html
URL: http://mmr.khu.ac.ir/article-1-2942-en.html
1- Alzahra University , r_alimohammadi@alzahra.ac.ir
2- Alzahra University
2- Alzahra University
Abstract: (832 Views)
Introduction
Regression analysis is a common method for modeling relationships between variables. Usually Ordinary Least Squares method is applied to estimate regression model parameters. These estimators are unbiased and appropriate when design matrix is nonsingular.
In presence of multicollinearity, design matrix is singular and Ordinary Least Squares estimates cannot be obtained. In this situation, other methods, such as Lasso, Ridge and Lars may be considered.
Other hand, in many fields such as medicine, number of variables is greater than the number of observations and usual methods such as Ordinary Least Squares are not proper and shrinkage methods, such as Lasso, Ridge and ... have a better performance to estimate regression model coefficients. In the shrinkage methods, tuning parameter plays an essential role in selecting variables and estimating parameters. Bridge shrinkage estimators is an estimator that can be obtained by changing its tuning parameter. In other words, Bridge method is the extension of Ridge and Lasso regression methods. Selecting the appropriate amount of tuning parameter is important. There are many studies on each of these methods under the assumed conditions. In this paper, performance of Bridge shrinkage estimators, such as Lasso and Ridge are compared with Lars and Ordinary Least Squares estimators in a simulation study.
Material and Methods
A simulation study is applied to compare performance of the regression methods Ridge, Lasso, Lars and Ordinary Least Squares. MSE criterion is applied to evaluate the method performance.
Statistical software R is applied for simulation and comparing the regression methods.
Results and discussion
In the presence of collinearity, Bridge regression estimators will result in appropriate estimators. These estimators are biased but their performance is better than unbiased estimators such as Ordinary Least Squares. Indeed, Bridge estimators have the best performance in the class of biased estimators.
Conclusion
In this article, Ridge and Lasso estimators as special cases of Bridge estimators are compared with Lasso and Ordinary Least Squares in a simulation study.
This study shows that under the supposed conditions, Ridge, Lasso and Lars have better action than Ordinary Least Squares method.
Lars method has the best performance and Ridge estimators is better than Lasso Regression.
Regression analysis is a common method for modeling relationships between variables. Usually Ordinary Least Squares method is applied to estimate regression model parameters. These estimators are unbiased and appropriate when design matrix is nonsingular.
In presence of multicollinearity, design matrix is singular and Ordinary Least Squares estimates cannot be obtained. In this situation, other methods, such as Lasso, Ridge and Lars may be considered.
Other hand, in many fields such as medicine, number of variables is greater than the number of observations and usual methods such as Ordinary Least Squares are not proper and shrinkage methods, such as Lasso, Ridge and ... have a better performance to estimate regression model coefficients. In the shrinkage methods, tuning parameter plays an essential role in selecting variables and estimating parameters. Bridge shrinkage estimators is an estimator that can be obtained by changing its tuning parameter. In other words, Bridge method is the extension of Ridge and Lasso regression methods. Selecting the appropriate amount of tuning parameter is important. There are many studies on each of these methods under the assumed conditions. In this paper, performance of Bridge shrinkage estimators, such as Lasso and Ridge are compared with Lars and Ordinary Least Squares estimators in a simulation study.
Material and Methods
A simulation study is applied to compare performance of the regression methods Ridge, Lasso, Lars and Ordinary Least Squares. MSE criterion is applied to evaluate the method performance.
Statistical software R is applied for simulation and comparing the regression methods.
Results and discussion
In the presence of collinearity, Bridge regression estimators will result in appropriate estimators. These estimators are biased but their performance is better than unbiased estimators such as Ordinary Least Squares. Indeed, Bridge estimators have the best performance in the class of biased estimators.
Conclusion
In this article, Ridge and Lasso estimators as special cases of Bridge estimators are compared with Lasso and Ordinary Least Squares in a simulation study.
This study shows that under the supposed conditions, Ridge, Lasso and Lars have better action than Ordinary Least Squares method.
Lars method has the best performance and Ridge estimators is better than Lasso Regression.
Keywords: Ridge Regression, Bridge Regression, Lasso Regression, LARS Regression, Tuning Parameter, Ordinary Least Squares Error Regression.
Type of Study: S |
Subject:
stat
Received: 2019/05/7 | Revised: 2022/11/16 | Accepted: 2020/11/14 | Published: 2022/05/21 | ePublished: 2022/05/21
Received: 2019/05/7 | Revised: 2022/11/16 | Accepted: 2020/11/14 | Published: 2022/05/21 | ePublished: 2022/05/21
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
