Mathematical Researches
پژوهش های ریاضی
mmr
Basic Sciences
http://mmr.khu.ac.ir
1
admin
2588-2546
2588-2554
10.61186/mmr
fa
jalali
1401
3
1
gregorian
2022
6
1
8
2
online
1
fulltext
fa
مقایسه شبیه سازی برآوردگرهای رگرسیونی بریج و لارس
a simulation comparison of Ridge regression estimators with Lars
آمار
stat
علمی پژوهشی کاربردی
S
تحلیل رگرسیون یکی از روشهای متداول آماری در مدلسازی روابط بین متغیرهاست. لذا در رگرسیون دو موضوع تعیین روابط بین متغیرها و تحلیل روابط حاصل مورد توجه قرار میگیرد.<br>
در مسائل با بعد بالا وقتی تعداد متغیرها بیشتر از تعداد مشاهدات است، روشهای معمول مانند رگرسیون کمترین توانهای دوم معمولی کارایی لازم را ندارند و روشهای انقباضی، ازجمله لاسو، ریج و ... از کارایی بهتری در برآورد ضرایب رگرسیونی برخوردار هستند. در این برآوردگرها پارامتر کنترل نقش اساسی در انتخاب متغیرهای تبیینی و برآورد پارامترها بازی میکند. برآوردگرهای انقباضی بریج، برآوردگری است که با تغییر پارامتر کنترل آن میتوان به برآوردگرهای ذکر شده دست یافت. در این مقاله برآوردگر انقباضی بریج از جمله لاسو و ریج را با برآوردگر لارس و کمترین توانهای دوم معمولی مقایسه کرده و کارایی آنها را با معیار میانگین توانهای دوم خطا مورد ارزیابی قرار میدهیم.<br>
<span style="font-size:11pt"><span style="line-height:normal"><span style="font-family:Calibri,sans-serif"><b><span style="font-size:10.0pt"><span new="" roman="" style="font-family:" times="">Introduction</span></span></b></span></span></span><br>
<span style="font-size:11pt"><span style="line-height:normal"><span style="font-family:Calibri,sans-serif"><span style="font-size:10.0pt"><span new="" roman="" style="font-family:" times="">Regression analysis is a common method for modeling relationships between variables. Usually Ordinary Least Squares method is applied to estimate regression model parameters. These estimators are unbiased and appropriate when design matrix is nonsingular. </span></span></span></span></span><br>
<span style="font-size:11pt"><span style="line-height:normal"><span style="font-family:Calibri,sans-serif"><span style="font-size:10.0pt"><span new="" roman="" style="font-family:" times="">In presence of multicollinearity, design matrix is singular and Ordinary Least Squares estimates cannot be obtained. In this situation, other methods‎, ‎such as Lasso‎, ‎Ridge‎ and ‎Lars may be considered.</span></span></span></span></span><br>
<span style="font-size:11pt"><span style="line-height:normal"><span style="font-family:Calibri,sans-serif"><span style="font-size:10.0pt"><span new="" roman="" style="font-family:" times="">Other hand, in many fields such as medicine, number of variables is greater than the number of observations‎ and usual methods such as Ordinary Least Squares are not proper and shrinkage methods‎, ‎such as Lasso‎, ‎Ridge‎ ‎and‎ ... ‎have a better performance to estimate regression model coefficients‎. ‎In the shrinkage methods‎, ‎tuning parameter plays an essential role in selecting variables and estimating parameters‎. ‎Bridge shrinkage estimators is an estimator that can be obtained by changing its tuning parameter‎. In other words, Bridge method is the extension of Ridge and Lasso regression methods. Selecting the appropriate amount of tuning parameter is important. There are many studies ‎on each of these methods under the assumed conditions. In this paper‎, performance of Bridge shrinkage estimators‎, ‎such as Lasso and Ridge‎ ‎are compared with Lars and Ordinary Least Squares estimators in a simulation study.</span></span></span></span></span><br>
<span style="font-size:11pt"><span style="line-height:normal"><span style="font-family:Calibri,sans-serif"><b><span style="font-size:10.0pt"><span new="" roman="" style="font-family:" times="">Material and Methods</span></span></b></span></span></span><br>
<span style="font-size:11pt"><span style="line-height:normal"><span style="font-family:Calibri,sans-serif"><span style="font-size:10.0pt"><span new="" roman="" style="font-family:" times="">A simulation study is applied to compare performance of the regression methods Ridge, Lasso, Lars and Ordinary Least Squares. MSE criterion is applied to evaluate the method performance.</span></span></span></span></span><br>
<span style="font-size:11pt"><span style="line-height:normal"><span style="font-family:Calibri,sans-serif"><span style="font-size:10.0pt"><span new="" roman="" style="font-family:" times="">Statistical software R is applied for simulation and comparing the regression methods.</span></span></span></span></span><br>
<span style="font-size:11pt"><span style="line-height:normal"><span style="font-family:Calibri,sans-serif"><b><span style="font-size:10.0pt"><span new="" roman="" style="font-family:" times="">Results and discussion</span></span></b></span></span></span><br>
<span style="font-size:11pt"><span style="line-height:normal"><span style="font-family:Calibri,sans-serif"><span style="font-size:10.0pt"><span new="" roman="" style="font-family:" times="">In the presence of collinearity, Bridge regression estimators will result in appropriate estimators. These estimators are biased but their performance is better than unbiased estimators such as Ordinary Least Squares. Indeed, Bridge estimators have the best performance in the class of biased estimators.</span></span></span></span></span><br>
<span style="font-size:11pt"><span style="line-height:normal"><span style="font-family:Calibri,sans-serif"><b><span style="font-size:10.0pt"><span new="" roman="" style="font-family:" times="">Conclusion</span></span></b></span></span></span><br>
<span style="font-size:11pt"><span style="line-height:normal"><span style="font-family:Calibri,sans-serif"><span style="font-size:10.0pt"><span new="" roman="" style="font-family:" times="">In this article, Ridge and Lasso estimators as special cases of Bridge estimators are compared with Lasso and Ordinary Least Squares in a simulation study. </span></span></span></span></span><br>
<span style="font-size:11pt"><span style="line-height:normal"><span style="font-family:Calibri,sans-serif"><span style="font-size:10.0pt"><span new="" roman="" style="font-family:" times="">This study shows that under the supposed conditions, Ridge, Lasso and Lars have better action than Ordinary Least Squares method.</span></span></span></span></span><br>
<span style="font-size:11pt"><span style="line-height:normal"><span style="font-family:Calibri,sans-serif"><span style="font-size:10.0pt"><span new="" roman="" style="font-family:" times="">Lars method has the best performance and Ridge estimators is better than Lasso Regression.</span></span></span></span></span><br>
رگرسیون کمترین توانهای دوم معمولی, رگرسیون ریج, رگرسیون بریج, رگرسیون لاسو, رگرسیون لارس, پارامتر کنترل.
Ridge Regression, Bridge Regression, Lasso Regression, LARS Regression, Tuning Parameter, Ordinary Least Squares Error Regression.
152
164
http://mmr.khu.ac.ir/browse.php?a_code=A-10-934-1&slc_lang=fa&sid=1
Roshanak
Alimohammadi
روشنک
علی محمدی
r_alimohammadi@alzahra.ac.ir
10031947532846005473
10031947532846005473
Yes
Alzahra University
دانشگاه الزهرا
Jaleh
Bahari
ژاله
بهاری
jaleh.bahari90@yahoo.com
10031947532846005474
10031947532846005474
No
Alzahra University
دانشگاه الزهرا