1
$\begingroup$

I have pandas dataframe with 8000 observations and with many different columns -some of them are dates and hours that were converted into dummies (get_dummies) and some are numerical,and y label which is what I want to predict that has two classes: 0 or 1:

>>>y   x1     x2     x3     x4   10   11   12   13
0  1  0.532  0.431  0.214  0.11  0    0    1    0
1  0  0.512  0.410  0.340  0.09  0    1    0    0
...

I have used random forest model in scikit learn in order to predict the y column (treatment) either is 1 or 0.

the results suprised me because when I print the accuracy I got -inf and I did not excpect that.

this is how I calcualte and run the rf:

X = df.drop(['y',],axis=1)
y = df['y'].astype(int)


#split the data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=123)

###Random forest

#function for evaluate:
def evaluate(model, test_features, test_labels):
    predictions = model.predict(test_features)
    errors = abs(predictions - test_labels)
    mape = 100 * np.mean(errors / test_labels)
    accuracy = 100 - mape
    print('Model Performance')
    print('Average Error: {:0.4f} degrees.'.format(np.mean(errors)))
    print('Accuracy = {:0.2f}%.'.format(accuracy))
    
    return accuracy

#random forest
from sklearn.ensemble import RandomForestClassifier
rfc = RandomForestClassifier()
rfc.fit(X_train,y_train)
predictions = rfc.predict(X_test)

base_model = RandomForestClassifier()
base_model.fit(X_train, y_train)
base_accuracy = evaluate(base_model, X_test, y_test)

>>>Model Performance
Average Error: 0.3290 degrees.
Accuracy = -inf%.

the confusion matrix looks like this: enter image description here

so my question is what that -inf% accuracy mean? where is the problem in the model?

$\endgroup$

1 Answer 1

1
$\begingroup$

Your implementation of accuracy is incorrect:

predictions = model.predict(test_features)
errors = abs(predictions - test_labels)
mape = 100 * np.mean(errors / test_labels)
accuracy = 100 - mape

You should use scikit-learn's accuracy function:

Something like:

from sklearn.metrics import accuracy_score

y_pred = model.predict(test_features)
accuracy = accuracy_score(test_labels, y_pred)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.