Empirical analysis of Ridge, Lasso, and Dropout regularizations

Yiming Wei

doi:10.1117/12.2639002

17 May 2022 Empirical analysis of Ridge, Lasso, and Dropout regularizations

Yiming Wei

Proceedings Volume 12259, 2nd International Conference on Applied Mathematics, Modelling, and Intelligent Computing (CAMMIC 2022); 122590H (2022) https://doi.org/10.1117/12.2639002
Event: 2nd International Conference on Applied Mathematics, Modelling, and Intelligent Computing, 2022, Kunming, China

Abstract

This paper analyzes the performance of Lasso, Ridge and Dropout Regression using different synthetic and real datasets covering classification and regression problems. The MNIST dataset displayed relatively satisfying results but little difference created by regularization. The Synthetic dataset displays a growing Mean Square Error(MSE) with greater signal to noise ratio as well as growing dimension of dataset. The comparison among the three regularizations reveals how Dropout regularization performs better with decreasing SNR, potentially related to the attribute of Dropout. However, the training model displayed mediocre results when tested upon real datasets and insignificant difference after the usage of regularizations, which could potentially be attributed to the depth of the training model being insufficient with the given data.

Citation Download Citation

Yiming Wei "Empirical analysis of Ridge, Lasso, and Dropout regularizations", Proc. SPIE 12259, 2nd International Conference on Applied Mathematics, Modelling, and Intelligent Computing (CAMMIC 2022), 122590H (17 May 2022); https://doi.org/10.1117/12.2639002

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available