# Questions tagged [lasso]

Least Absolute Shrinkage and Selection Operator (LASSO) regression, is a regularization technique used in regression cases where the model overfits or there is high multi-collinearity.

34 questions
Filter by
Sorted by
Tagged with
23 views

### Elegant way to plot the L2 regularization path of logistic regression in python?

Trying to plot the L2 regularization path of logistic regression with the following code (an example of regularization path can be found in page 65 of the ML textbook Elements of Statistical Learning ...
19 views

### Do I have to remove features with pairwise correlation even if I am doing a regularized logistic regression?

Normally we would remove features that have high pairwise correlation with another feature before performing regression. But is this step necessary if I am applying L2 regularized logistic regression (...
24 views

### Can I rescale TF matrix or TF-IDF matrix using StandardScaler prior to Logisitc Lasso regression?

I am trying to use Logistic Lasso to classify documents as 1 or 0. I've tried using both the TF matrix and TF-IDF matrix representations of the documents as my predictors. I've found that if I use the ...
38 views

### What's the correct cost function for Linear Regression

As we all know the cost function for linear regression is: Where as when we use Ridge Regression we simply add lambda*slope**2 but there I always seee the below as cost function of linear Regression ...
31 views

there is one question for me. I want to know that how Adam or Adagrad treat l1-norm regularization in loss-function? (e.g. Lasso) I know that l1-norm is not differentiable function at zero but we can ...
16 views

### How to handle both the categorical and ordinal features in a single data sets?

I was practicing Lasso regression with the SPARCS hospital dataset. There are two kinds of features in the dataset: Categorical features like location of the hospital, demographics of patients, etc. ...
11 views

### How to set coefficient limit in lasso regression in Python?

I'm working on a regression problem where I want to use Lasso model. With the help of Lasso and LassoCV, we can provide different alpha values and get the best parameter and coefficients however I ...
29 views

### Should I use or tune reg_lambda or reg_alpha hyperparameters when using a tree booster in XGBoost

XGBoost has 3 types of boosters: tree boosters (gbtree, dart) linear booster (gbliner) ...
13 views

### Linear models: Imputing missing not at random

This question is a continuation of a similar question for linear models instead of Tree-based model. Given that linear models (e.g. lasso, ridge, Linear regression, elastic net, etc.) can't handle ...
73 views

### Difference between PCA and regularisation

Currently, I am confusing about PCA and regularisation. I wonder what is the difference between PCA and regularisation: particularly lasso (L1) regression? Seems both of them can do the feature ...
93 views

### Lack of standardization in Kaggle jupyter notebooks when using lasso/ridge?

I've recently started using Kaggle, and I've noticed that for a lot of these jupyter notebooks written by others, when they use Ridge/Lasso, they don't standardize the non-categorical numerical ...
26 views

### Is there an point to using LassoCV with cross_val_score

I've seen some jupyter notebooks that seem to combine LassoCV with cross_val_score, and I'm confused what the point is. Usually ...
192 views

### sklearn gridsearch lasso regression: find specific number of coefficients

I am using GridSearchCV and Lasso regression in order to fit a dataset composed out of Gaussians. I keep this example similar to this tutorial. My goal is to find the best solution with a restricted ...
43 views

### Lasso regression not getting better without random features

First of all, I'm new to lasso regression, so sorry if this feels stupid. I'm trying to build a regression model and wanted to use lasso regression for feature selection as I have quite a few features ...
288 views

### How to remove features from a sklearn pipeline after it has already been fitted?

Background: I have created a basic modeling workflow in sklearn that utilizes sklearn's pipeline object. There are some preprocessing steps within the pipeline, and the last step of the pipeline is to ...
80 views

### Lasso Regression for Feature Importance saying almost every feature is unimportant?

I have a metric (RevenueSoFar) that is a great predictor of my target FinalRevenue as you'd expect - it is a metric where we tend to get 90-95% of revenue so far on day 1 and then it can increase over ...
24 views

### Interpreting machine learning coefficients

My dog show predictive tool is having some trouble with its neural net. Broadly, I start with a couple of factors--age, weight, height, breed (which is a set of dummy variables), a subjective cuteness ...
20 views

### Should I normalize the dependent variable in a penalized linear regression model?

When I compute penalized regression on the data without normalizing using the glmnet package in R, the lambda values and RMSE generated in lasso, ridge, and elastic net are unreasonably large. The ...
235 views

### What is the meaning of the sparsity parameter

Sparse methods such as LASSO contain a parameter $\lambda$ which is associated with the minimization of the $l_1$ norm. Higher the value of $\lambda$ ($>0$) means that more coefficients will be ...
59 views

### how Lasso regression helps to shrinks the coefficient to zero and why ridge regression dose not shrink the coefficient to zero?

How Lasso regression helps feature selection of model by making the coefficient to zero? , I could see few below with below diagram ,can any please explain in simple terms how to corelate below ...
301 views

### regarding lasso.score in lasso modeling using scikit-learn

I once saw the following code segment of using lasso model based on scikit-learn ...
28 views

### Generating artificial data to extend learning set

I have dataset containing 42 instances(X) and one final Y on which i want to perform LASSO regression.All are continuous and numerical. As the sample size small, I wish to extend it. I am kind of ...
688 views

### How do standardization and normalization impact the coefficients of linear models?

One benefit of creating a linear model is that you can look at the coefficients the model learns and interpret them. For example, you can see which features have the most predictive power and which do ...
297 views

### How is learning rate calculated in sklearn Lasso regression?

I was applying different regression models to Kaggle Housing dataset for advanced regression. I am planning to test out lasso, ridge and elastic net. However, none of these models have learning rate ...
16 views

### Lasso stricter with more data

I am currently analyzing investment strategies, and have implemented a backtest accordingly. This essentially means that I predict returns each month by using all prior historical data. Consequently, ...
101 views

### For a square matrix of data, I achieve $R^2=1$ for Linear Regression and $R^2=0$ for Lasso. What's the intuition behind?

For a square matrix of random data, N columns and N rows. I am fitting two models, linear regression and Lasso. For the linear regression, I achieve a perfect score in train set, while in the Lasso I ...
20 views

### When should we start using stacking of models?

I am solving a Kaggle contest and my single model has reached score of 0.121, I'd like to know when to start using ensembling/stacking to improve the score. I used lasso and xgboost and there ...
237 views

### Normalisation results in R^2 score of 0 - Lasso regression

I am running a regression analysis on a 7000 row dataset with a train/test split of 70%/30%. I am using one variable X to predict a variable ...
40 views

### Scikit-compatible network lasso implementation

Is anyone aware of a scikit-compatible network Lasso (nLasso) implementation? These papers have source code as well: D. Hallac, J. Leskovec, and S. Boyd, “Network lasso: Clustering and ...
66 views

### LASSO remaining features for different penalisation

I am using the sklearn LASSOCV function and I am changing the penalisation parameter in order to adjust the number of features killed off. For example for $\alpha = 0.01$ I have 55 features remaining ...
246 views

### Why does Lasso behave "erratically" when the number of features is greater than the number of training instances?

From the book "Hands-on Machine Learning with Scikit-Learn and TensorFlow 2nd edition" chapter 4: In general, Elastic Net is preferred over Lasso since Lasso may behave erratically when the ...
77 views

### Why do you need to use group lasso with categorical variables?

From what I've read you should you use group lasso to either discard the dummy encoded variables (of the category) or use all of them. If you use normal lasso then some of the variables in the group ...