Which regularizer to use to get a sparse set of regression parameters?

Question

I am doing a regression and I want to use the regularizer that will be the most useful to get a sparse set of parameters.

Which regularizer should I use ? Cardinality? maximum value ? Sum of absolute values ? Euclidean norm ?

Answer 1

The most common sparse regularizer is sum of absolute values (so-called Lasso regression). With carefully chosen penalty coefficient, it makes some of less useful parameters exactly zero.

Cardinality penalty exactly imposes sparsity, but it cannot be combined with gradient descent, and usually requires combinatorial optimization. Simply put, it is slow to apply.

Maximum value and Euclidean norm do not affect sparsity at all.

You can find more details in the wikipeda article.