I have fit n LASSO Regressions on n different data sets (the 'datasets' object is an R list of length n where each element is a data.table which is a light and fast data frame from the data.table package) using the enet() function from the elastic net package in R using the code below:
set.seed(11)
L.fits <- lapply(X = datasets, function(i)
elasticnet::enet(x = as.matrix(dplyr::select(i,
starts_with("X"))),
y = i$Y, lambda = 0, normalize = FALSE))
I have then determined which of the 30 candidate variables for each of the n data sets the LASSO fit on that data set selects and saved those via the following lines of code:
## This stores and prints out all of the regression
## equation specifications selected by LASSO when called.
L.Coeffs <- lapply(X = L.fits,
function(i) predict(i,
x = as.matrix(dplyr::select(i, starts_with("X"))),
s = 0.1, mode = "fraction",
type = "coefficients")[["coefficients"]])
# This object stores just the names of the variables selected by LASSO.
Variables.Selected <- lapply(L.Coeffs, function(i) names(i[i > 0]))
After running this all on the last 50 datasets, this is what I get:
> tail(Variables_Selected, n = 5)
[[1]]
[1] "X1" "X3" "X20"
[[2]]
[1] "X1" "X20"
[[3]]
[1] "X1" "X9" "X10" "X12" "X16" "X17"
[[4]]
[1] "X1"
[[5]]
[1] "X1" "X23"
[[6]]
[1] "X2" "X4"
I first tried to replicate the selections/results using the glmnet function from the package of the same name with the code below:
set.seed(11) # to ensure replicability
L_fits <- lapply(datasets, function(i)
glmnet(x = as.matrix(select(i, starts_with("X"))),
y = i$Y, alpha = 0))
L_coefs = L.fits |>
Map(f = \(model) coef(model, s = .1))
Variables_Selected <- L_coefs |>
Map(f = \(matr) matr |> as.matrix() |>
as.data.frame() |> filter(s1 != 0) |> rownames())
And all of the code above does run, but when I then run Variables_Selected, I get all 30 candidate variables back for all of the LASSOs:
> tail(Variables.Selected, n = 3)
[[1]]
[1] "(Intercept)" "X1" "X2" "X3" "X4" "X5"
[7] "X6" "X7" "X8" "X9" "X10" "X11"
[13] "X12" "X13" "X14" "X15" "X16" "X17"
[19] "X18" "X19" "X20" "X21" "X22" "X23"
[25] "X24" "X25" "X26" "X27" "X28" "X29"
[31] "X30"
[[2]]
[1] "(Intercept)" "X1" "X2" "X3" "X4" "X5"
[7] "X6" "X7" "X8" "X9" "X10" "X11"
[13] "X12" "X13" "X14" "X15" "X16" "X17"
[19] "X18" "X19" "X20" "X21" "X22" "X23"
[25] "X24" "X25" "X26" "X27" "X28" "X29"
[31] "X30"
[[3]]
[1] "(Intercept)" "X1" "X2" "X3" "X4" "X5"
[7] "X6" "X7" "X8" "X9" "X10" "X11"
[13] "X12" "X13" "X14" "X15" "X16" "X17"
[19] "X18" "X19" "X20" "X21" "X22" "X23"
[25] "X24" "X25" "X26" "X27" "X28" "X29"
[31] "X30"
And when I tried using a 2nd alternative option to replicate my findings with the lars function from the package of that name, I get valid results, but they are not identical to when I used the enet function. The code I used to fit them are included here:
set.seed(11) # to ensure replicability
LASSO.Lars.fits <- lapply(X = datasets, function(i)
lars(x = as.matrix(select(i, starts_with("X"))),
y = i$Y, type = "lasso"))
LASSO.Lars.Coeffs <- lapply(LASSO.Lars.fits,
function(i) predict(i,
x = as.matrix(dplyr::select(i, starts_with("X"))),
s = 0.1, mode = "fraction",
type = "coefficients")[["coefficients"]])
IVs.Selected.by.Lars <- lapply(LASSO.Lars.Coeffs, function(i) names(i[i > 0]))
The last few of lars's selections are included here:
> tail(IVs.Selected.by.Lars)
[[1]]
[1] "X3" "X4" "X20"
[[2]]
[1] "X2" "X3" "X20"
[[3]]
[1] "X4" "X8" "X9" "X10" "X12" "X17"
[[4]]
[1] "X2" "X3" "X7"
[[5]]
[1] "X3" "X10" "X23"
[[6]]
[1] "X1"
How can I either get my glmnet function to actually make some selections here or determine which of the sets of selected variables (if any!) by the other two are the valid set of variables selected by LASSO?
p.s. By the way the Monte Carlo Simulation used to create the synthetic data sets, I know both how many of the 30 candidate regressors in them are true aka "structural" variables, and which of them they are, and this information is stored in another R list of length n called Structural_Variables. The final 6 elements of this list are:
> tail(Structural_Variables)
[[1]]
[1] "X2" "X3" "X4" "X8" "X9" "X10" "X16" "X17" "X18" "X19" "X20" "X22" "X23" "X25" "X28"
[[2]]
[1] "X1" "X3" "X4" "X7" "X8" "X13" "X14" "X18" "X19" "X20" "X21" "X22" "X23" "X24" "X30"
[[3]]
[1] "X3" "X4" "X8" "X9" "X10" "X12" "X13" "X16" "X18" "X20" "X22" "X24" "X26" "X28" "X29"
[[4]]
[1] "X1" "X2" "X4" "X6" "X7" "X8" "X9" "X11" "X13" "X14" "X15" "X18" "X26" "X27" "X28"
[[5]]
[1] "X1" "X3" "X8" "X9" "X10" "X11" "X12" "X19" "X20" "X22" "X23" "X24" "X26" "X29" "X30"
[[6]]
[1] "X1" "X2" "X4" "X6" "X9" "X10" "X11" "X16" "X17" "X19" "X21" "X22" "X23" "X24" "X26"
This list is then used to compare to the lists Variables_Selected, Variables.Selected, and IVs.Selected.by.Lars to see how well each of these different methods of running LASSO performed.
