# Which metric to use to evaluate prediction problem

The product manager wants to know if you can develop a model to predict the number of views a listing will receive based on the boat's features. She would consider using your model if, on average, the predictions were only 50% off of the true number of views a listing would receive.

The data used in this: https://www.kaggle.com/datasets/karthikbhandary2/boat-sales

I tried using the RandomForestRegressor() model along with RandomizedSearchCV.

param = [
{'n_estimators': [100, 200, 300, 400, 450, 500, 600, 700, 800, 900, 1000],
'max_depth': [3, 4, 6, 8, 10, 12,14,16,18,20],
'max_leaf_nodes': [15, 20, 25,30,35,40]},
]

rf = RandomForestRegressor(random_state=42)
rs_rf = RandomizedSearchCV(rf, param, cv = 4, n_jobs = -1, verbose = 1)
rs_rf.fit(X_train, y_train)


I tried using RMSE like this:

rmse = np.sqrt(mean_squared_error(y_test, y_pred))
print(rmse)


I got the result as 483887.72666891955

My question is whether I am doing this right or not?

• If you want to see how much the prediction is off from the actual value as a percentage you should look into using the mean average percentage error (MAPE). Commented Mar 19, 2022 at 11:39
• How can I do that? So RMSE is not the metric to use? Commented Mar 19, 2022 at 11:40
• If you are mainly interested in the model's error as a percentage of the actual value then no. Commented Mar 19, 2022 at 11:58
• So I should go with MAPE? And How can I implement it? Can you submit it as an answer?? Commented Mar 19, 2022 at 12:04
• Now it is returning 1.2265745506841434, after using the mean_average_percentage_error. Commented Mar 19, 2022 at 13:02

If you want to see how much the prediction is off from the actual value as a percentage you should look into using the mean average percentage error (MAPE). The scikit-learn library has this loss implemented in the metrics package. You can simply call the mean_absolute_percentage_error function instead of the mean_squared_error you're currently using:
mean_absolute_percentage_error(y_true, y_pred)