Questions tagged [ridge-regression]

A regularization method for regression models that shrinks coefficients towards zero.

Filter by
Sorted by
Tagged with
0 votes
0 answers
29 views

Help with multinomial logistic regression

I am a data science student and have the opportunity to work on an article regrading cardiac arrests in our country. For now I performed the multinomial regression model and I also plan on doing a ...
Ni Vaznu's user avatar
0 votes
0 answers
4 views

With infinite observations, would the weights resulting from ridge regression be the same as simple linear regression?

As the number of observations approaches infinity, do the weights of a linear regression approach the weights of a linear regression with L2 penalty?
BigMistake's user avatar
0 votes
0 answers
22 views

Regression model in Python for engineering equipment end of life prediction - improvements and suggestions

I am trying to build a ML prediction model that uses the calculated remaining part life, machine power consumption, and other related process parameters specific to the equipment in question, to ...
amy_hislop's user avatar
0 votes
0 answers
83 views

Why does SGDRegressor with partial_fit not converge to the same R2 as RidgeCV

I have a dataframe of about 200 features and 1M rows that I can train a RidgeCV model and get an R2 of about 0.01 I'd like to scale up my training to 5M or 10M rows but that won't fit in memory for me ...
nxtrad00r's user avatar
0 votes
0 answers
19 views

Classification problem with a numerical variable that uses a special (high) value to indicate a qualitatively different status

I have a classification problem where I need to predict an outcome based on 20+ variables, some categorical, some numerical. One of the numerical variables is 'dlast' - which is the number of days ...
Ido Sarig's user avatar
  • 101
0 votes
0 answers
11 views

How to use this data set for spatial regression?

I want to graph how much a customer spends by region and have hotspots for high spending regions. Here is an example of the csv file.
ChickenJoe's user avatar
0 votes
1 answer
27 views

How to extract MSEP or RMSEP from lassoCV?

I'm doing lasso and ridge regression in R with the package chemometrics. With ridgeCV it is easy to extract the SEP and MSEP values by ...
Sally's user avatar
  • 5
0 votes
1 answer
155 views

Why are my ridge regression coefficients completely different from ordinary linear regression coefficients in MATLAB?

I am attempting to implement my own Ridge Regression algorithm and I am trying to achieve similar coefficients found in a MATLAB tutorial on regression. Specifically, on the MATLAB tutorial page you ...
user1068636's user avatar
1 vote
1 answer
285 views

What's the correct cost function for Linear Regression

As we all know the cost function for linear regression is: Where as when we use Ridge Regression we simply add lambda*slope**2 but there I always seee the below as cost function of linear Regression ...
Chris_007's user avatar
  • 193
1 vote
1 answer
417 views

The effect of the λ in the Ridge regression

Why by increasing value of λ in Ridge estimator the slope of the line is decreasing? How exactly λ affects to the y = kx + b?
Dablup's user avatar
  • 11
3 votes
2 answers
3k views

Constraining linear regressor parameters in scikit-learn?

I'm using sklearn.linear_model.Ridge to use ridge regression to extract the coefficients of a polynomial. However, some of the coefficients have physical ...
awho's user avatar
  • 31
1 vote
1 answer
67 views

Why we take $\alpha\sum B_j^2$ as penalty in Ridge Regression?

$$RSS_{RIDGE}=\sum_{i=1}^n(\hat{y_i}-y_i)^2+\alpha\sum_{i=1}^nB_j^2$$ Why we are taking $\alpha\sum B_j^2$ as a penalty here? We are adding this term for minimizing variance in Machine Learning Model. ...
Ujjwal Kar's user avatar
-1 votes
1 answer
106 views

Lack of standardization in Kaggle jupyter notebooks when using lasso/ridge?

I've recently started using Kaggle, and I've noticed that for a lot of these jupyter notebooks written by others, when they use Ridge/Lasso, they don't standardize the non-categorical numerical ...
student010101's user avatar
1 vote
1 answer
292 views

Do the benefits of ridge regression diminish with larger datasets?

I have a question about ridge regression and about its benefits (relative to OLS) when the datasets are big. Do the benefits of ridge regression disappear when the datasets are larger (e.g. 50,000 vs ...
Rocky the Owl's user avatar
1 vote
2 answers
645 views

What is the meaning of the sparsity parameter

Sparse methods such as LASSO contain a parameter $\lambda$ which is associated with the minimization of the $l_1$ norm. Higher the value of $\lambda$ ($>0$) means that more coefficients will be ...
Sm1's user avatar
  • 521
2 votes
3 answers
406 views

how Lasso regression helps to shrinks the coefficient to zero and why ridge regression dose not shrink the coefficient to zero?

How does Lasso regression help with feature selection of model by making the coefficient shrink to zero? I could see few below with below diagram. Can any please explain in simple terms how to ...
star's user avatar
  • 1,431
1 vote
0 answers
18 views

what other metrics can i use to estimate quality of the model predicting income range - interval estimation task?

I trained a model that predicts customer's income given the features: age, declared income number of oustanding instalment, overdue total amount active credit limit, total credit limit total amount ...
wando's user avatar
  • 111
2 votes
2 answers
3k views

How do standardization and normalization impact the coefficients of linear models?

One benefit of creating a linear model is that you can look at the coefficients the model learns and interpret them. For example, you can see which features have the most predictive power and which do ...
codeananda's user avatar
1 vote
1 answer
123 views

Why is Regularization after PCA or Factor Analysis a bad idea?

I have done Factor Analysis on my data and applied various machine learning models on it. I particularly find it giving high MSE value for Ridge and Lasso Regression compared to other models. I want ...
Poo's user avatar
  • 23
1 vote
1 answer
839 views

How is learning rate calculated in sklearn Lasso regression?

I was applying different regression models to Kaggle Housing dataset for advanced regression. I am planning to test out lasso, ridge and elastic net. However, none of these models have learning rate ...
Aman Krishna's user avatar
1 vote
1 answer
626 views

Is there a reference data set for ridge regression?

In order to test an algorithm, I am looking for a reference data set for ridge regression in research papers. Kind of like the equivalent of MNIST but for regression.
Marie's user avatar
  • 23
4 votes
2 answers
2k views

Extremely high MSE/MAE for Ridge Regression(sklearn) when the label is directly calculated from the features

Edit: Removing TransformedTargetRegressor and adding more info as requested. Edit2: There were 18K rows where the relation did not hold. I'm sorry :(. After ...
RAbraham's user avatar
  • 187
3 votes
1 answer
500 views

Does ridge regression always reduce coefficients by equal proportions?

Below is an excerpt from the book Introduction to statistical learning in R, (chapter-linear model selection and regularization) "In ridge regression, each least squares coefficient estimate is ...
Preetham_tsp's user avatar
1 vote
1 answer
175 views

Dividing the weights obtained on an already standardized data set by the standard deviation of the features? (Ridge regression)

I'm trying to understand a code snippet from my lecture on Machine Learning (see the code below). It extracts the mean and standard deviation of the features and uses them to 'normalize' (...
infinite789's user avatar
3 votes
3 answers
8k views

Can ridge regression be used for feature selection?

I'm trying to figure out whether using Ridge Regression for regularization can be used to cause a more sparse hypothesis however to me it seems like ridge will never actually bring any coefficients to ...
crommy's user avatar
  • 197
3 votes
1 answer
199 views

Does it matter whether we put regularization parameter ($C$) with error or weight term in Kernel ridge regression?

Kernel ridge regression associate a regularization parameter $C$ with weight term ($\beta$): $\text{Minimize}: {KRR}=C\frac{1}{2} \left \|\beta\right\|^{2} + \frac{1}{2}\sum_{i=1}^{\mathcal{N}}\left\|...
Chandan Gautam's user avatar
1 vote
1 answer
337 views

How to improve Regression Model with High Training Performance and Low Test Performance

I am performing regression analysis on some data. I keep getting very high training score and low test score. My code is below, what can i do to enhance it? Thank you in advance. ...
tsumaranaina's user avatar
4 votes
2 answers
3k views

What does a negative coefficient of determination mean for evaluating ridge regression?

Judging by the negative result being displayed from my ridge.score() I am guessing that I am doing something wrong. Maybe someone could point me in the right ...
Ethan's user avatar
  • 1,635