I am getting a DataFrame.dtypes error while following the last steps of this tutorial.

Here is my code:

import xgboost as xgb
regr = xgb.XGBRegressor(colsample_bytree=0.2,

regr.fit(train_new, label_df)

And this is the error I am receiving:

ValueErrorTraceback (most recent call last)
<ipython-input-64-010296d611e9> in <module>()
     12                        silent=1)
---> 14 regr.fit(train_new, label_df)

D:\anacodna\lib\site-packages\xgboost\sklearn.pyc in fit(self, X, y, sample_weight, eval_set, eval_metric, early_stopping_rounds, verbose, xgb_model, sample_weight_eval_set, callbacks)
    358                                    missing=self.missing, nthread=self.n_jobs)
    359         else:
--> 360             trainDmatrix = DMatrix(X, label=y, missing=self.missing, nthread=self.n_jobs)
    362         evals_result = {}

D:\anacodna\lib\site-packages\xgboost\core.pyc in __init__(self, data, label, missing, weight, silent, feature_names, feature_types, nthread)
    378         data, feature_names, feature_types = _maybe_pandas_data(data,
    379                                                                 feature_names,
--> 380                                                                 feature_types)
    382         data, feature_names, feature_types = _maybe_dt_data(data,

D:\anacodna\lib\site-packages\xgboost\core.pyc in _maybe_pandas_data(data, feature_names, feature_types)
    237         msg = """DataFrame.dtypes for data must be int, float or bool.
    238                 Did not expect the data types in fields """
--> 239         raise ValueError(msg + ', '.join(bad_fields))
    241     if feature_names is None:

ValueError: DataFrame.dtypes for data must be int, float or bool.
                Did not expect the data types in fields Alley, Condition2, Electrical, GarageType, GarageYrBlt, Heating, LandContour, LandSlope, LotShape, MiscFeature, PavedDrive, RoofMatl, Street, Utilities
  • $\begingroup$ Looks like you have data other than numbers in your dataset. Convert your categorical to dummies or label encoding or something to convert everything to numbers.it says the fields mentioned in the error are not numbers. $\endgroup$ Jan 24 '20 at 15:37
  • $\begingroup$ Hello , yes I had categorial data but it was converted using LabelEncoder $\endgroup$
    – Kasia
    Jan 24 '20 at 15:46
  • $\begingroup$ Can you print a snippet of your data and also your data frame.dtypes please ? $\endgroup$ Jan 24 '20 at 16:04
  • $\begingroup$ Hello I am using this tutorial I its well explained, and all clarifications as well as code is written on this site hackerearth.com/practice/machine-learning/… I am doing everything the same way to try it by myself. I am sorry I dont know how to add snippet to a comment. but here is the dataset kaggle.com/c/house-prices-advanced-regression-techniques. I am just wondering if I have just bassically copied the whole think from tutorial site for study purpose, why I am getting this error, while there is no error in tutorial $\endgroup$
    – Kasia
    Jan 24 '20 at 18:02
  • $\begingroup$ just use this: ``` regr.fit(train_new.values, label_df) ``` $\endgroup$ Dec 27 '20 at 19:43

Check the values of train_new. You'll see that the columns mentionned are not of the expected types.

Another suggestion, i'm not sure of xgboost's handling of nulls. Might be that in those columns aswell.

  • $\begingroup$ actually i am facing the same error, when in fact i have one hot encoded all my categorical variables. Also there are no nan values in my data. When I run the model, everything runs fine but when I deploy it using Flask, it gives me this error. Here is the link to my question. I would love to know your thoughts on it:- datascience.stackexchange.com/questions/99696/… $\endgroup$
    – spectre
    Aug 6 at 17:53

Perform Label encoding for the features like Alley, Condition2, etc (mentioned in your error). For example:

train_new['Electrical'] = le.fit_transform(train_new['Electrical'].astype(str))

The above is the snippet for your reference. Do not apply .astype(str) for Condition2 only. Excluding that u need to apply the line that I gave u for all attributes (mentioned in error). I hope this solves your error.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.