I'm trying to use L1 regularization to select features in XGBoost classifier. However, I don't see any example code on how to specify the penalty of l1.

This is how I do in sklearn's LogisticRegression.

C = [10, 1, .1, .001]

for c in C:
    clf = LogisticRegression(penalty='l1', C=c)
    clf.fit(X_train, y_train)
    print('C:', c)
    print('Training accuracy:', clf.score(X_train, y_train))

How should I specify penalty and C in XGBoost?


L2 and L1 regularization are controlled via the lambda (=reg_lambda) and alpha (=reg_alpha) parameter respectively. Higher values of alpha mean more L1 regularization. See the documentation here.

  • $\begingroup$ Hi, so if we are using alpha, we don't have to say penalty='l1', correct? $\endgroup$
    – Aerin
    Apr 2 '18 at 21:24
  • $\begingroup$ that's right... $\endgroup$
    – oW_
    Apr 3 '18 at 15:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.