1
$\begingroup$

What are the ways to speed up the fit of the model on large files (more than 200 mb)? I tried to divide into pieces (chunks) or use dask but the effect is unsatisfactory. I use 16GB RAM and i7 2.2GHz processor 4 core. It remains to buy a cloud service or use a better computer? My clasiffier

clf = xgb.XGBClassifier(n_estimators=500,
                        n_jobs=4,
                        max_depth=9,
                        learning_rate=0.05,
                        subsample=0.7,
                        colsample_bytree=0.7,
                        missing=-999, 
                        gamma = 0.1)
# Number of variable > 400
# Number of rows > 200000
Improve this question
$\endgroup$
5
  • $\begingroup$ what type of model? $\endgroup$
    – Peter
    Jul 18, 2019 at 19:00
  • $\begingroup$ I tried with xboost. In general its binary classification problem. $\endgroup$
    – fuwiak
    Jul 18, 2019 at 19:02
  • $\begingroup$ How many observations and variables do you have in your dataset? How many variables are categorical and how many levels they have? How do you encode categorical variables? Note that naive one-hot encoding might significantly increase the number of variables passed to xgboost. What parameters do you use (e.g. max depth, number of trees, sample rates and learning rate)? What is your current run times? P.S. You might want to include this information directly in your question. $\endgroup$
    – aivanov
    Jul 18, 2019 at 20:48
  • $\begingroup$ What about categorical variables? Can you include this information as well? $\endgroup$
    – aivanov
    Jul 19, 2019 at 7:20
  • $\begingroup$ less than 50 variables. $\endgroup$
    – fuwiak
    Jul 19, 2019 at 10:35

2 Answers 2

Reset to default
2
$\begingroup$

One thing you could try is to use a "normal" Logit, as it is computationally not very expensive. Using Lasso, Elastic Net, or Ridge can yield good results (often similar to boosting) shown here.

Here is a recent code example for Logit/Lasso.

If you would like to stick with boosting, you may check LightGBM as it tends to be faster. XGBoost tends to be "heavy" in terms of data handling. LightGBM (as the name indicates) aims at resolving this problem and I have had good experiences so far.

Here is the link to the the LightGBM docs.

Improve this answer
$\endgroup$
5
  • $\begingroup$ I will check LightGBM. What about techniques? I looking a more general solution. $\endgroup$
    – fuwiak
    Jul 18, 2019 at 19:57
  • $\begingroup$ I guess splitting the file etc will not give you an advantage if you mean this by saying „techniques“. I think optimizing data handling is key. Also check Logit as a baseline. It should be fast. $\endgroup$
    – Peter
    Jul 18, 2019 at 20:07
  • $\begingroup$ Oh... you could also go on Kaggle if it is only for one project. There you can run a Kernel on GPU for free. kaggle.com $\endgroup$
    – Peter
    Jul 18, 2019 at 20:12
  • $\begingroup$ Fine, but if its for job purpose? $\endgroup$
    – fuwiak
    Jul 18, 2019 at 20:14
  • $\begingroup$ Ähm... maybe not due to data protection issues. $\endgroup$
    – Peter
    Jul 18, 2019 at 20:34
0
$\begingroup$

You can try the following two techniques or even a combination of both

  1. in case you are using one-hot encoding for categorical variables and have variables with many levels (i.e. categorical variables with high cardinality), you could first lump rare levels together and then encode or just use another approach to encode categorical variables (e.g. target encoding).

  2. use only a small subset of observations (rows) to identify the most relevant variables (columns) and then refit the model using only these variables on the entire dataset. If the quality of the final model would be not satisfactory you’ll need to include more variables.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.