-1
$\begingroup$

I am doing a regression and I want to use the regularizer that will be the most useful to get a sparse set of parameters.

Which regularizer should I use ? Cardinality? maximum value ? Sum of absolute values ? Euclidean norm ?

$\endgroup$

1 Answer 1

4
$\begingroup$

The most common sparse regularizer is sum of absolute values (so-called Lasso regression). With carefully chosen penalty coefficient, it makes some of less useful parameters exactly zero.

Cardinality penalty exactly imposes sparsity, but it cannot be combined with gradient descent, and usually requires combinatorial optimization. Simply put, it is slow to apply.

Maximum value and Euclidean norm do not affect sparsity at all.

You can find more details in the wikipeda article.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.