Skip to main content
OverflowAI is here! AI power for your Stack Overflow for Teams knowledge community. Learn more

Questions tagged [lasso]

Least Absolute Shrinkage and Selection Operator (LASSO) regression, is a regularization technique used in regression cases where the model overfits or there is high multi-collinearity.

Filter by
Sorted by
Tagged with
0 votes
0 answers
9 views

Elastic Net alpha value using GLMNET 4.1-8

Is it a valid method to “brute force” the alpha value for an elastic net? What I mean is trying alpha = .1, .2, .3, .4 and so on to 1.0 and looking at the highest R-squared value of each and choosing ...
user162172's user avatar
0 votes
0 answers
23 views

Adaptive Lasso Coefficient Weights

I'm trying to understand how the Adaptive part of Adaptive Lasso works. I understand that theoretically, the weights for zero coefficients are inflated to infinity. But can someone explain this ...
user162172's user avatar
0 votes
0 answers
22 views

Sample Size for Adaptive Lasso

Be gentle, I'm learning here. I have a fairly simple adaptive lasso regression that I'm trying to test for a minimum sample size. I used cross-validated mean squared error as the "score" of ...
JRDubbleu's user avatar
0 votes
0 answers
12 views

Interpreting large discrepancies between Specificities & the # of Extraneous Variable Models selected by a variable selection algorithm

I am going to preface my question by saying that this problem of interpretation I have run into is in the context of me doing my part as a collaborator on a statistical learning paper for the first ...
Marlen's user avatar
  • 167
0 votes
0 answers
16 views

merging for LASSO for categorical variables

s0 (Intercept) 0.2991246444 animalDog -0.0006095736 mfMale . age -0....
user392987's user avatar
0 votes
0 answers
13 views

Optimal method for predicting outcome from many additive, correlated, and sparse features?

Suppose I have many vectors which can take on any of three values, 0, 1, 2. These vectors affect an outcome being predicted, Y. Vectors add together: a vector "A" of the value 2 has twice ...
BigMistake's user avatar
0 votes
0 answers
25 views

Standardization of LASSO Regression

currently I am using the lasso regression to identify a energy function. So there is an input which is lets say x and I am creating a library of nonlinear functions of x. Those functions should ...
user150587's user avatar
0 votes
2 answers
52 views

The ideal function in R for fit fitting n LASSO Regressions on n data sets

As part of a statistical learning research paper I am collaborating on, I am running/fitting two hundred sixty thousand different LASSO Regressions on the same number of different randomly generated ...
Marlen's user avatar
  • 167
0 votes
1 answer
65 views

Having problems replicating the results of a series of N LASSOs fit to N datasets in R

I have fit n LASSO Regressions on n different data sets (the 'datasets' object is an R list of length n where each element is a data.table which is a light and fast data frame from the data.table ...
Marlen's user avatar
  • 167
1 vote
1 answer
54 views

A good 4th Benchmark method to compare the performance of a novel Variable Selection Algorithm being evaluated

I am collaborating on a research project with a respected econometrician as a graduate student (although only in an MS program, not PhD program mind you) exploring the properties and comparing the ...
Marlen's user avatar
  • 167
0 votes
2 answers
2k views

Which standardization technique to use for Lasso regression?

I am fitting a Lasso Regression to do feature selection in my dataset. I have seen it is common practice to use StandardScaler to standardize the dataset. However, given that the distribution of my ...
StephM's user avatar
  • 11
0 votes
1 answer
25 views

A simple way to store the factors selected by (BE) Stepwise Regression n run on N datasets via lapply, a For Loop then an lapply, or a function?

I am currently doing research with a coauthor and collaborator comparing a new optimal model selection procedure he has proposed via Monte Carlo Simulation of the new procedure vs 2 benchmarks, LASSO &...
Marlen's user avatar
  • 167
1 vote
2 answers
635 views

Group lasso and feature selection

I have a dataset with a lot of categorical variables and a binary target variable and I want to put it to an svm. I converted the categorical variables to dummy variables and since my observations are ...
user avatar
0 votes
1 answer
58 views

How to compare between two methods of multivariate to filling NA

In the Titanic dataset, I performed two methods to fill Age NA. The first one is regression using Lasso: ...
Husam Khiry's user avatar
0 votes
1 answer
28 views

How to extract MSEP or RMSEP from lassoCV?

I'm doing lasso and ridge regression in R with the package chemometrics. With ridgeCV it is easy to extract the SEP and MSEP values by ...
Sally's user avatar
  • 5
1 vote
1 answer
75 views

Why is the L2 penalty squared but the L1 penalty isn't in elastic-net regression?

There was some data set I worked with which I wanted to solve non negative least squares (NNLS) on and I wanted a sparse model. After a bit of experiementing I found that what worked the best for me ...
Tomer Wolberg's user avatar
0 votes
1 answer
1k views

Why is gridsearchCV.best_estimator_.score giving me r2_score even if I mentioned MAE as my main scoring metric?

I have a lasso regression model with the following definition : ...
Echo's user avatar
  • 113
1 vote
1 answer
153 views

Is it possible to explain why Lasso models eliminated certain coefficient?

Is it possible to understand why Lasso models eliminated specific coefficients?. During the modelling, many of the highly correlated features in data is being eliminated by Lasso regression. Is it ...
NAS_2339's user avatar
  • 243
0 votes
1 answer
352 views

Accessing regression coefficients when using MultiOutputRegressor

I am working on a multioutput (nr. targets: 2) regression task. The original data has a huge dimensionality (p>>n, i.e. there are far more predictors than ...
lazarea's user avatar
  • 299
1 vote
0 answers
50 views

Predicting single floats based on set of 2 feature arrays each of 100 values

I am trying to predict audio to video desynchronization based on set of two arrays of lenght 100 which consist of coresponding audio and video samples. The problem is that my labels are single floats (...
fdrobiazg's user avatar
1 vote
1 answer
52 views

Why are we not checking the significance of the coefficients in Lasso and elastic net models

As far as I know, we don't check the coefficient significance in Lasso and elasticnet models. Is it because insignificant feature coefficients will be driven to zero in these models?. Does that mean ...
NAS_2339's user avatar
  • 243
1 vote
1 answer
499 views

Elegant way to plot the L2 regularization path of logistic regression in python?

Trying to plot the L2 regularization path of logistic regression with the following code (an example of regularization path can be found in page 65 of the ML textbook Elements of Statistical Learning ...
lostwanderer's user avatar
0 votes
1 answer
56 views

Do I have to remove features with pairwise correlation even if I am doing a regularized logistic regression?

Normally we would remove features that have high pairwise correlation with another feature before performing regression. But is this step necessary if I am applying L2 regularized logistic regression (...
lostwanderer's user avatar
2 votes
0 answers
255 views

Can I rescale TF matrix or TF-IDF matrix using StandardScaler prior to Logisitc Lasso regression?

I am trying to use Logistic Lasso to classify documents as 1 or 0. I've tried using both the TF matrix and TF-IDF matrix representations of the documents as my predictors. I've found that if I use the ...
Patrick Steele's user avatar
1 vote
1 answer
376 views

What's the correct cost function for Linear Regression

As we all know the cost function for linear regression is: Where as when we use Ridge Regression we simply add lambda*slope**2 but there I always seee the below as cost function of linear Regression ...
Chris_007's user avatar
  • 193
1 vote
2 answers
104 views

What (linear) model is common practice to use on sample size of 500 with 26 features?

I have a training data set of 500 people and 26 features and I'm trying to develop a regression model. A possibility is to derive more features of course. I'm considering the following models: Linear ...
CDS's user avatar
  • 11
0 votes
1 answer
593 views

How to set coefficient limit in lasso regression in Python?

I'm working on a regression problem where I want to use Lasso model. With the help of Lasso and LassoCV, we can provide different alpha values and get the best parameter and coefficients however I ...
Ashwini's user avatar
3 votes
1 answer
800 views

Difference between PCA and regularisation

Currently, I am confusing about PCA and regularisation. I wonder what is the difference between PCA and regularisation: particularly lasso (L1) regression? Seems both of them can do the feature ...
Crazy's user avatar
  • 133
-1 votes
1 answer
109 views

Lack of standardization in Kaggle jupyter notebooks when using lasso/ridge?

I've recently started using Kaggle, and I've noticed that for a lot of these jupyter notebooks written by others, when they use Ridge/Lasso, they don't standardize the non-categorical numerical ...
student010101's user avatar
0 votes
0 answers
861 views

sklearn gridsearch lasso regression: find specific number of coefficients

I am using GridSearchCV and Lasso regression in order to fit a dataset composed out of Gaussians. I keep this example similar to this tutorial. My goal is to find the best solution with a restricted ...
Felix's user avatar
  • 1
1 vote
1 answer
155 views

Lasso regression not getting better without random features

First of all, I'm new to lasso regression, so sorry if this feels stupid. I'm trying to build a regression model and wanted to use lasso regression for feature selection as I have quite a few features ...
Onur Ece's user avatar
0 votes
1 answer
2k views

How to remove features from a sklearn pipeline after it has already been fitted?

Background: I have created a basic modeling workflow in sklearn that utilizes sklearn's pipeline object. There are some preprocessing steps within the pipeline, and the last step of the pipeline is to ...
DataScienceRick's user avatar
1 vote
1 answer
145 views

Lasso Regression for Feature Importance saying almost every feature is unimportant?

I have a metric (RevenueSoFar) that is a great predictor of my target FinalRevenue as you'd expect - it is a metric where we tend to get 90-95% of revenue so far on day 1 and then it can increase over ...
James's user avatar
  • 113
2 votes
1 answer
244 views

Interpreting machine learning coefficients

My dog show predictive tool is having some trouble with its neural net. Broadly, I start with a couple of factors--age, weight, height, breed (which is a set of dummy variables), a subjective cuteness ...
gibbsabroad's user avatar
1 vote
2 answers
755 views

What is the meaning of the sparsity parameter

Sparse methods such as LASSO contain a parameter $\lambda$ which is associated with the minimization of the $l_1$ norm. Higher the value of $\lambda$ ($>0$) means that more coefficients will be ...
Sm1's user avatar
  • 541
2 votes
3 answers
632 views

how Lasso regression helps to shrinks the coefficient to zero and why ridge regression dose not shrink the coefficient to zero?

How does Lasso regression help with feature selection of model by making the coefficient shrink to zero? I could see few below with below diagram. Can any please explain in simple terms how to ...
star's user avatar
  • 1,471
0 votes
1 answer
2k views

regarding lasso.score in lasso modeling using scikit-learn

I once saw the following code segment of using lasso model based on scikit-learn ...
user288609's user avatar
1 vote
1 answer
45 views

Generating artificial data to extend learning set

I have dataset containing 42 instances(X) and one final Y on which i want to perform LASSO regression.All are continuous and numerical. As the sample size small, I wish to extend it. I am kind of ...
rik's user avatar
  • 11
2 votes
2 answers
3k views

How do standardization and normalization impact the coefficients of linear models?

One benefit of creating a linear model is that you can look at the coefficients the model learns and interpret them. For example, you can see which features have the most predictive power and which do ...
codeananda's user avatar
1 vote
1 answer
910 views

How is learning rate calculated in sklearn Lasso regression?

I was applying different regression models to Kaggle Housing dataset for advanced regression. I am planning to test out lasso, ridge and elastic net. However, none of these models have learning rate ...
Aman Krishna's user avatar
1 vote
0 answers
21 views

Lasso stricter with more data

I am currently analyzing investment strategies, and have implemented a backtest accordingly. This essentially means that I predict returns each month by using all prior historical data. Consequently, ...
John's user avatar
  • 111
4 votes
1 answer
120 views

For a square matrix of data, I achieve $R^2=1$ for Linear Regression and $R^2=0$ for Lasso. What's the intuition behind?

For a square matrix of random data, N columns and N rows. I am fitting two models, linear regression and Lasso. For the linear regression, I achieve a perfect score in train set, while in the Lasso I ...
Carlos Mougan's user avatar
2 votes
1 answer
64 views

When should we start using stacking of models?

I am solving a Kaggle contest and my single model has reached score of 0.121, I'd like to know when to start using ensembling/stacking to improve the score. I used lasso and xgboost and there ...
thewhitetulip's user avatar
3 votes
1 answer
774 views

Normalisation results in R^2 score of 0 - Lasso regression

I am running a regression analysis on a 7000 row dataset with a train/test split of 70%/30%. I am using one variable X to predict a variable ...
atom's user avatar
  • 31
1 vote
0 answers
52 views

Scikit-compatible network lasso implementation

Is anyone aware of a scikit-compatible network Lasso (nLasso) implementation? These papers have source code as well: D. Hallac, J. Leskovec, and S. Boyd, “Network lasso: Clustering and ...
Pörripeikko's user avatar
4 votes
2 answers
83 views

LASSO remaining features for different penalisation

I am using the sklearn LASSOCV function and I am changing the penalisation parameter in order to adjust the number of features killed off. For example for $\alpha = 0.01$ I have 55 features remaining ...
prax1telis's user avatar
4 votes
2 answers
856 views

Why does Lasso behave "erratically" when the number of features is greater than the number of training instances?

From the book "Hands-on Machine Learning with Scikit-Learn and TensorFlow 2nd edition" chapter 4: In general, Elastic Net is preferred over Lasso since Lasso may behave erratically when the ...
Moaz Ashraf's user avatar
2 votes
0 answers
158 views

Why do you need to use group lasso with categorical variables?

From what I've read you should you use group lasso to either discard the dummy encoded variables (of the category) or use all of them. If you use normal lasso then some of the variables in the group ...
Ferus's user avatar
  • 121
0 votes
2 answers
366 views

What happens when scikit-learn does a Lasso Model?

I have started an MLS course. As a beginner and non-mathematician it has been hard. I am trying to understand the exercise about Lasso Models. I have done Lasso models on R-cran, but this is my first ...
user avatar
1 vote
0 answers
57 views

Lasso implementation Drawback

Recently I've been trying to implement Lasso by myself in R, not using the "glmnet" package, and based on an article by Tibshirani I wrote a raw code to implement coordinate descent method, and it ...
DiaryofNewton's user avatar