1
$\begingroup$

I built machine learning model for Ridge,lasso, elastic net and linear regression, for that I used gridsearch for the parameter tuning, i want to know how give value range for **params Ridge ** below code? example consider alpha parameter there i uses for alpha 1,0.1,0.01,0.001,0.0001,0 but i haven't idea how this values determine each models.(ridge/lasso/elastic) can some one explain these things?

 from sklearn.linear_model import Ridge
    ridge_reg = Ridge()
    from sklearn.model_selection import GridSearchCV
    params_Ridge = {'alpha': [1,0.1,0.01,0.001,0.0001,0] , "fit_intercept": [True, False], "solver": ['svd', 'cholesky', 'lsqr', 'sparse_cg', 'sag', 'saga']}
    Ridge_GS = GridSearchCV(ridge_reg, param_grid=params_Ridge, n_jobs=-1)
    Ridge_GS.fit(x_train,y_train)
    Ridge_GS.best_params_
Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

I am not sure if I understand your question correctly, but in your model, you are tuning your "alpha" parameter, you have a range from 1 to 0. e.g. (1 -> 0.1 -> 0.01 -> 0.001 -> 0.0001 -> ... -> 0)

The grid search will evaluate each algorithm (SVD, CHOLESKY,...) with each possible value of your "alpha" parameter. It will define the score for each alpha parameter (eg. accuracy / auc). The score metric depends on your estimator / choice

Documentation:

https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html

Improve this answer
$\endgroup$
5
  • $\begingroup$ problem is how i choose this 'alpha': [1,0.1,0.01,0.001,0.0001,0] array, how determine values for alpha? is possible give values for alpha like 1,5,10,22 etc?? for a what is method to determine values for hyper parameter?? $\endgroup$
    – randunu galhena
    Feb 4, 2020 at 11:27
  • 1
    $\begingroup$ It's what you choose yourself. The grid search will tell you which alpha is the best. You can choose whatever alpha you want. But typically, alpha are around 0.1, 0.01, 0.001 ... The grid search will help you to define what alpha you should use; eg the alpha with the best score. So if you choose more values, you can do ranges from 100 -> 10 -> 1 -> 0.1. And see how the score changes dependent on these values. If it goes down from 100, to 10, and goes up at 1. You can choose again and zoom in on the values from 10 to 1 until you find the optimal value $\endgroup$
    – aze45sq6d
    Feb 4, 2020 at 11:53
  • $\begingroup$ I got your point. that meant when selecting values for alpha, there no any restriction I can put any values in side array then, grid search select best values from array is it correct?? and it is does not depend on model type whether lasso/ ridge or elastic net. for this three models when use grid search I can give same alpha values separately??( alpha value array does not depend on model) $\endgroup$
    – randunu galhena
    Feb 4, 2020 at 12:13
  • 1
    $\begingroup$ Well, theoretically there are no restrictions. But technically, you have the restriction of time. Using more hyperparameters takes longer to evaluate. The alpha values you define there are used for all the algorithms you are evaluating them on. But you can also change this. $\endgroup$
    – aze45sq6d
    Feb 4, 2020 at 12:16
  • 1
    $\begingroup$ But you can also separate this by implementing a "search space" and "pipeline". This way, you can define each list of hyper parameters separately for each algo. Please check the documentation: scikit-learn.org/stable/modules/grid_search.html $\endgroup$
    – aze45sq6d
    Feb 4, 2020 at 12:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.