# High error machine learning regressor algorithm in Python - XGBOOST Regressor

I have a dataframe with real state data from florida, it includes single apartments and buildings data:

'TRUE_SITE_CITY': The city where the building is. variable: Miami, Aventura...;
'CONDO_FLAG': If it is a condominium or not, variable: yes/no;
'BEDROOM_COUNT': Number of total bethrooms, variable: integuer,
'BUILDING_actual_AREA': The area of the entire building, or apartment in the case that there are only one apartment or house. variable: integuer;
'FLOOR_COUNT': Number of the floors that the building has;
'DOR_CODE_CUR': the type of the building. Variable: categorical;
'UNIT_COUNT': Number of apartments or houses in the building. Variable: integuer;
'YEAR_BUILT': Year that the building or house or apartment was build: Variable: categorical;
'public_transport_min_distance': I have calculated the nearest stations of the public transport;
'Price': The variable that I want to predict.Variable: integer.


I have done an exploratory data analysis and I have dropped some data that has null values and some data that was incorrect. Also I have dropped the values with outliers.

The basic statistics of the price column (targeted column):

I have checked the categorical features and they have enough variables in each one to keep they in the model.

I have done a pipeline to make a one hot encoder for the categorical values and a standard standardisation for the numerical values. In it I have include a XGBOOST regression:

from xgboost import XGBRegressor
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import cross_validate
from sklearn import metrics
from sklearn import preprocessing, feature_extraction
from sklearn.pipeline import Pipeline
from sklearn import preprocessing, feature_extraction
from sklearn.pipeline import make_pipeline, make_union
from mlxtend.feature_selection import ColumnSelector
from sklearn.preprocessing import StandardScaler
from category_encoders import OneHotEncoder

x_numeric = df_x[['BEDROOM_COUNT','BATHROOM_COUNT',
'HALF_BATHROOM_COUNT', 'FLOOR_COUNT','UNIT_COUNT','public_transport_min_distance','BUILDING_actual_AREA']]

x_categorical = df_x[['TRUE_SITE_CITY','CONDO_FLAG','YEAR_BUILT']]

categorical_col = x_categorical.columns

numeric_col = x_numeric.columns

estimator_pipeline = Pipeline([
])

score2 = cross_validate(estimator_pipeline, X= df_x, y= df_y, scoring=scoring,return_train_score=False, cv=5,n_jobs=2)


But I am obtaining a high error. The mean value of the price is almost 200.000 and the error that I obtain is:

I have done feature selection using RFE but I obtain a high error as well.

Also I have run it doing RandomizedSearchCV

from sklearn.model_selection import RandomizedSearchCV

params = {"estimator__learning_rate"    : [0.05, 0.10, 0.15, 0.20, 0.25, 0.30 ] ,
"estimator__max_depth"        : [ 3, 4, 5, 6, 8, 10, 12, 15],
"estimator__min_child_weight" : [ 1, 3, 5, 7 ],
"estimator__gamma"            : [ 0.0, 0.1, 0.2 , 0.3, 0.4 ],
"estimator__colsample_bytree" : [ 0.3, 0.4, 0.5 , 0.7 ] }

random_search = RandomizedSearchCV(
estimator=estimator_pipeline,
param_distributions=params, cv=5, refit=True,
scoring="neg_mean_squared_error", n_jobs= 3,
return_train_score=True,
n_iter=50)


But I obtain a similar error value.

What could I do?