2
$\begingroup$

I'm a newbie in data science. I'm working on a regression problem. I'm getting 2.5 MAPE. 400 MAE 437000 MSE. As my MAPE is quite low but why I'm getting high MSE and MAE? This is the link to my data

from sklearn.metrics import mean_absolute_error 
from sklearn.metrics import mean_squared_error
from sklearn.preprocessing import Normalizer
import matplotlib.pyplot as plt
def mean_absolute_percentage_error(y_true, y_pred): 
    y_true, y_pred = np.array(y_true), np.array(y_pred)
    return np.mean(np.abs((y_true - y_pred) / y_true)) * 100

import pandas as pd
from sklearn import preprocessing

features=pd.read_csv('selectedData.csv')
import numpy as np
from scipy import stats
print(features.shape)
features=features[(np.abs(stats.zscore(features)) < 3).all(axis=1)]
target = features['SYSLoad']
features= features.drop('SYSLoad', axis = 1)
names=list(features)

for i in names:
    x=features[[i]].values.astype(float)
    min_max_scaler = preprocessing.MinMaxScaler()
    x_scaled = min_max_scaler.fit_transform(x)
    features[i]=x_scaled

Selecting the target Variable which want to predict and for which we are finding feature imps

import numpy as np
print(features.shape)
print(features.describe())
from sklearn.model_selection import train_test_split
train_input, test_input, train_target, test_target = 
train_test_split(features, target, test_size = 0.25, random_state = 42)
trans=Normalizer().fit(train_input);
train_input=Normalizer().fit_transform(train_input);
test_input=trans.fit_transform(test_input);

n=test_target.values;
test_targ=pd.DataFrame(n);

from sklearn.svm import SVR
svr_rbf = SVR(kernel='poly', C=10, epsilon=10,gamma=10)
y_rbf = svr_rbf.fit(train_input, train_target);
predicted=y_rbf.predict(test_input);
plt.figure
plt.xlim(300,500);
print('Total Days For training',len(train_input)); print('Total Days For 
Testing',len(test_input))
plt.ylabel('Load(MW) Prediction 3 '); plt.xlabel('Days'); 
plt.plot(test_targ,'-b',label='Actual'); plt.plot(predicted,'-r',label='POLY 
kernel ');
plt.gca().legend(('Actual','RBF'))
plt.title('SVM')
plt.show();


test_target=np.array(test_target)
print(test_target)
MAPE=mean_absolute_percentage_error(test_target,predicted);
print(MAPE);
mae=mean_absolute_error(test_target,predicted)
mse=mean_squared_error(test_target, predicted)
print(mae);
print(mse);
print(test_target);
print(predicted);
$\endgroup$
2
$\begingroup$

I'll be honest, I haven't thoroughly checked your code. However, I can see that the range of values of your dataset is approx [0,12000]. As an engineer, I see that:

  1. sqrt(MSE) = sqrt(437000) = 661 units.
  2. MAE = 400 units.
  3. MAPE = 2.5 which means that MAE can be up to 0.025*12000= 250 units.

All three cases show similar magnitude of error, so I wouldn't say that "MAPE is quite low but you're getting high mse and MAE".

Those 3 values explain the results from similar yet different perspectives. Keep in mind, if the values were all the same, there would have been no need for all 3 of those metrics to exist :)

$\endgroup$
  • $\begingroup$ Thank you and what does r-square metrics shows? $\endgroup$ – imtiaz ul Hassan Feb 19 '19 at 16:25
  • $\begingroup$ I believe Wikipedia has a very nice explanation :) en.wikipedia.org/wiki/Coefficient_of_determination $\endgroup$ – pcko1 Feb 19 '19 at 21:49
  • $\begingroup$ @pcko1 thank you for your answer. Is it possible to elaborate? I didn't understand what you've meat for 1 to 3. For instance, what do you mean by MAE = 400 units? $\endgroup$ – Media Mar 21 '19 at 22:04
  • $\begingroup$ well MAE stands for Mean Absolute Error and OP mentioned that he gets "2.5 MAPE, 400 MAE, 437000 MSE". I just tried to make a back-of-the-envelope evaluation of the magnitute of those values, which seem reasonable given his dataset :) as for "units", this refers to the physical units of the problem, whatever they might be $\endgroup$ – pcko1 Mar 21 '19 at 22:14
1
$\begingroup$

You are stating something that is by definition the case. A Mean Absolute Percentage Error (MAPE) is typically a small number expressed in percentage, hopefully in the single digit. Meanwhile the Mean Squared Error (MSE) and Mean Absolute Error) are expressed in square of units and units respectively. If your units are > 1, the MSE could get easily very large on a relative basis compared to the MAE. And, the MAE could also be a lot larger than the MAPE. This is just like saying a nominal number is a lot larger than its log or natural logs. Your three error measures are measured on pretty different scale.

They just bring some perspective to how well your model fit your data. Depending on the circumstances or context one error measure may be more relevant than the other.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.