0
$\begingroup$

I am training a Decision Tree Regressor on a relatively small data. The dimensions of my train and test sets are (34164, 10) and (8514, 10). Here is the relevant code:

y = np.log(data2['price'])
data2.drop(['price'], axis = 1, inplace = True)

num_cols = [cname for cname in data2.columns if data2[cname].dtype in ['int64', 'float64']]
cat_cols = [cname for cname in data2.columns if data2[cname].dtype == 'object']

num_trans = SimpleImputer(strategy = 'mean')
cat_trans = Pipeline(steps = [('impute', SimpleImputer(strategy = 'most_frequent')), 
                          ('onehotencode', OneHotEncoder(handle_unknown = 'ignore'))])

preproc = ColumnTransformer(transformers = [('cat', cat_trans, cat_cols), 
                                        ('num', num_trans, num_cols)])


dtr_model = DecisionTreeRegressor(random_state = 69, criterion = 'mae')

dtr_pipe = Pipeline(steps = [('preproc', preproc), ('model', dtr_model)])

train_x, test_x, train_y, test_y = train_test_split(data2, y, test_size = 0.2, 
random_state=69)


# BASELINE MODEL
cross_dtr_score = -1 * cross_val_score(dtr_pipe, train_x, train_y, cv = 5,
                                    n_jobs = -1, scoring = 'neg_mean_absolute_error')
base_dtr_score = cross_dtr_score.mean()

The problem is it is taking too long to run, even for the baseline model. This is the first time I am facing this problem as usually any kind of tree based model does not take this long. Also the train and test dataset is not huge. So why is it taking such a long time to run even for something as simple as a baseline model? By such a long time I mean more than 15 minutes!

$\endgroup$
9
  • $\begingroup$ Apparently you have some categorical features. How many dimensions after one-hot encoding? I would suspect that there are too many, in this case you should simplify your categorical variables probably. $\endgroup$
    – Erwan
    Jul 23 at 23:22
  • $\begingroup$ @Erwan after OHE I get 306 features. And I cannot further simply the features without loosing valuable info. I guess the problem has something to do with Pipelines. I think they are making the computation slow but I'm not sure. $\endgroup$
    – spectre
    Jul 24 at 5:22
  • 1
    $\begingroup$ Then that's not the reason, it's a very reasonable number. I don't have any other idea, except that I'd be curious to know if the processing time is spent more in the preprocessing or the actual training. But I don't know if this is possible to check. $\endgroup$
    – Erwan
    Jul 24 at 9:01
  • 1
    $\begingroup$ I don't think so: the only part which can run in parallel is the CV and it's already done. I don't think the preprocessing can be distributed in sklearn, but I'm not sure. Anyway the first thing would be to identify which part is taking time. $\endgroup$
    – Erwan
    Jul 24 at 13:23
  • 1
    $\begingroup$ No there's no problem with MSE. It gives higher error values than MAE because the error is squared, which means that it penalizes more strongly systems which make larger errors compared to MAE. If you don't have any reason to choose one or the other both are fine. $\endgroup$
    – Erwan
    Aug 1 at 11:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.