Questions tagged [lasso]

Least Absolute Shrinkage and Selection Operator (LASSO) regression, is a regularization technique used in regression cases where the model overfits or there is high multi-collinearity.

Filter by
Sorted by
Tagged with
1
vote
1answer
23 views

Elegant way to plot the L2 regularization path of logistic regression in python?

Trying to plot the L2 regularization path of logistic regression with the following code (an example of regularization path can be found in page 65 of the ML textbook Elements of Statistical Learning ...
1
vote
1answer
19 views

Do I have to remove features with pairwise correlation even if I am doing a regularized logistic regression?

Normally we would remove features that have high pairwise correlation with another feature before performing regression. But is this step necessary if I am applying L2 regularized logistic regression (...
2
votes
0answers
24 views

Can I rescale TF matrix or TF-IDF matrix using StandardScaler prior to Logisitc Lasso regression?

I am trying to use Logistic Lasso to classify documents as 1 or 0. I've tried using both the TF matrix and TF-IDF matrix representations of the documents as my predictors. I've found that if I use the ...
1
vote
1answer
38 views

What's the correct cost function for Linear Regression

As we all know the cost function for linear regression is: Where as when we use Ridge Regression we simply add lambda*slope**2 but there I always seee the below as cost function of linear Regression ...
0
votes
0answers
31 views

Can Adagrad or Adam be used in loss function with l1-norm regularization?

there is one question for me. I want to know that how Adam or Adagrad treat l1-norm regularization in loss-function? (e.g. Lasso) I know that l1-norm is not differentiable function at zero but we can ...
0
votes
0answers
16 views

How to handle both the categorical and ordinal features in a single data sets?

I was practicing Lasso regression with the SPARCS hospital dataset. There are two kinds of features in the dataset: Categorical features like location of the hospital, demographics of patients, etc. ...
0
votes
1answer
11 views

How to set coefficient limit in lasso regression in Python?

I'm working on a regression problem where I want to use Lasso model. With the help of Lasso and LassoCV, we can provide different alpha values and get the best parameter and coefficients however I ...
0
votes
0answers
29 views

Should I use or tune `reg_lambda` or `reg_alpha` hyperparameters when using a tree booster in XGBoost

XGBoost has 3 types of boosters: tree boosters (gbtree, dart) linear booster (gbliner) ...
0
votes
0answers
13 views

Linear models: Imputing missing not at random

This question is a continuation of a similar question for linear models instead of Tree-based model. Given that linear models (e.g. lasso, ridge, Linear regression, elastic net, etc.) can't handle ...
3
votes
1answer
73 views

Difference between PCA and regularisation

Currently, I am confusing about PCA and regularisation. I wonder what is the difference between PCA and regularisation: particularly lasso (L1) regression? Seems both of them can do the feature ...
-1
votes
1answer
93 views

Lack of standardization in Kaggle jupyter notebooks when using lasso/ridge?

I've recently started using Kaggle, and I've noticed that for a lot of these jupyter notebooks written by others, when they use Ridge/Lasso, they don't standardize the non-categorical numerical ...
0
votes
0answers
26 views

Is there an point to using LassoCV with `cross_val_score`

I've seen some jupyter notebooks that seem to combine LassoCV with cross_val_score, and I'm confused what the point is. Usually ...
0
votes
0answers
192 views

sklearn gridsearch lasso regression: find specific number of coefficients

I am using GridSearchCV and Lasso regression in order to fit a dataset composed out of Gaussians. I keep this example similar to this tutorial. My goal is to find the best solution with a restricted ...
1
vote
1answer
43 views

Lasso regression not getting better without random features

First of all, I'm new to lasso regression, so sorry if this feels stupid. I'm trying to build a regression model and wanted to use lasso regression for feature selection as I have quite a few features ...
0
votes
1answer
288 views

How to remove features from a sklearn pipeline after it has already been fitted?

Background: I have created a basic modeling workflow in sklearn that utilizes sklearn's pipeline object. There are some preprocessing steps within the pipeline, and the last step of the pipeline is to ...
1
vote
1answer
80 views

Lasso Regression for Feature Importance saying almost every feature is unimportant?

I have a metric (RevenueSoFar) that is a great predictor of my target FinalRevenue as you'd expect - it is a metric where we tend to get 90-95% of revenue so far on day 1 and then it can increase over ...
2
votes
1answer
24 views

Interpreting machine learning coefficients

My dog show predictive tool is having some trouble with its neural net. Broadly, I start with a couple of factors--age, weight, height, breed (which is a set of dummy variables), a subjective cuteness ...
0
votes
0answers
20 views

Should I normalize the dependent variable in a penalized linear regression model?

When I compute penalized regression on the data without normalizing using the glmnet package in R, the lambda values and RMSE generated in lasso, ridge, and elastic net are unreasonably large. The ...
1
vote
2answers
235 views

What is the meaning of the sparsity parameter

Sparse methods such as LASSO contain a parameter $\lambda$ which is associated with the minimization of the $l_1$ norm. Higher the value of $\lambda$ ($>0$) means that more coefficients will be ...
0
votes
2answers
59 views

how Lasso regression helps to shrinks the coefficient to zero and why ridge regression dose not shrink the coefficient to zero?

How Lasso regression helps feature selection of model by making the coefficient to zero? , I could see few below with below diagram ,can any please explain in simple terms how to corelate below ...
0
votes
1answer
301 views

regarding lasso.score in lasso modeling using scikit-learn

I once saw the following code segment of using lasso model based on scikit-learn ...
1
vote
0answers
28 views

Generating artificial data to extend learning set

I have dataset containing 42 instances(X) and one final Y on which i want to perform LASSO regression.All are continuous and numerical. As the sample size small, I wish to extend it. I am kind of ...
2
votes
2answers
688 views

How do standardization and normalization impact the coefficients of linear models?

One benefit of creating a linear model is that you can look at the coefficients the model learns and interpret them. For example, you can see which features have the most predictive power and which do ...
1
vote
1answer
297 views

How is learning rate calculated in sklearn Lasso regression?

I was applying different regression models to Kaggle Housing dataset for advanced regression. I am planning to test out lasso, ridge and elastic net. However, none of these models have learning rate ...
1
vote
0answers
16 views

Lasso stricter with more data

I am currently analyzing investment strategies, and have implemented a backtest accordingly. This essentially means that I predict returns each month by using all prior historical data. Consequently, ...
4
votes
1answer
101 views

For a square matrix of data, I achieve $R^2=1$ for Linear Regression and $R^2=0$ for Lasso. What's the intuition behind?

For a square matrix of random data, N columns and N rows. I am fitting two models, linear regression and Lasso. For the linear regression, I achieve a perfect score in train set, while in the Lasso I ...
2
votes
1answer
20 views

When should we start using stacking of models?

I am solving a Kaggle contest and my single model has reached score of 0.121, I'd like to know when to start using ensembling/stacking to improve the score. I used lasso and xgboost and there ...
1
vote
1answer
237 views

Normalisation results in R^2 score of 0 - Lasso regression

I am running a regression analysis on a 7000 row dataset with a train/test split of 70%/30%. I am using one variable X to predict a variable ...
1
vote
0answers
40 views

Scikit-compatible network lasso implementation

Is anyone aware of a scikit-compatible network Lasso (nLasso) implementation? These papers have source code as well: D. Hallac, J. Leskovec, and S. Boyd, “Network lasso: Clustering and ...
4
votes
2answers
66 views

LASSO remaining features for different penalisation

I am using the sklearn LASSOCV function and I am changing the penalisation parameter in order to adjust the number of features killed off. For example for $\alpha = 0.01$ I have 55 features remaining ...
4
votes
2answers
246 views

Why does Lasso behave "erratically" when the number of features is greater than the number of training instances?

From the book "Hands-on Machine Learning with Scikit-Learn and TensorFlow 2nd edition" chapter 4: In general, Elastic Net is preferred over Lasso since Lasso may behave erratically when the ...
2
votes
0answers
77 views

Why do you need to use group lasso with categorical variables?

From what I've read you should you use group lasso to either discard the dummy encoded variables (of the category) or use all of them. If you use normal lasso then some of the variables in the group ...
0
votes
2answers
171 views

What happens when scikit-learn does a Lasso Model?

I have started an MLS course. As a beginner and non-mathematician it has been hard. I am trying to understand the exercise about Lasso Models. I have done Lasso models on R-cran, but this is my first ...
1
vote
0answers
46 views

Lasso implementation Drawback

Recently I've been trying to implement Lasso by myself in R, not using the "glmnet" package, and based on an article by Tibshirani I wrote a raw code to implement coordinate descent method, and it ...