Large mean squared error in sklearn regressors

Question

I'm a beginner in machine learning and I want to build a model to predict the price of houses. I prepared a dataset by crawling a local housing website and it consists 1000 samples and only 4 features (latitude, longitude, area and number of rooms).

I tried RandomForestRegressorand LinearSVR models in sklearn, but I can't train the model properly and the MSE is super high.

MSE almost equals 90,000,000 (the true values of prices' range are between 5,000,000 - 900,000,000)

Here is my code:

import numpy as np
from sklearn.svm import LinearSVR
import pandas as pd
import csv
from sklearn.preprocessing import normalize
from sklearn.model_selection import train_test_split

df = pd.read_csv('dataset.csv', index_col=False)
X = df.drop('price', axis=1)

X_data = X.values
Y_data = df.price.values
X_train, X_test, Y_train, Y_test = train_test_split(X_data, Y_data, test_size=0.2, random_state=5)

rgr = RandomForestRegressor(n_estimators=100)
svr = LinearSVR()

rgr.fit(X_train, Y_train)
svr.fit(X_train, Y_train)

MSEs = cross_val_score(estimator=rgr,
                         X=X_train,
                         y=Y_train,
                         scoring='mean_squared_error',
                         cv=5)

MSEsSVR = cross_val_score(estimator=svr,
                         X=X_train,
                         y=Y_train,
                         scoring='mean_squared_error',
                         cv=5)

MSEs *= -1
RMSEs = np.sqrt(MSEs)

print("Root mean squared error with 95% confidence interval:")
print("{:.3f} (+/- {:.3f})".format(RMSEs.mean(), RMSEs.std()*2))
print("")

Is the problem with my dataset and count of features? How can I build a prediction model with this type of dataset?

Answer 1

that's possibly due to poor parameter tuning.
Try reducing C for SVR and increasing n_estimators for RFR.

A nice approach is to gridsearch through the parameter, and plot the metric result.

Another thing that might help is to normalize the parameters (sklearn.preprocessing.StandardScaler) and to remove the skew from the target (usually log transform or 1/target transform works better)

Answer 2

I removed 20 percent of most expensive houses from dataset and divided the prices by 1.000.000 and the result got much better.