I applied this random forest algorithm to predict a specific crime type. The example I took from this article here.

import pandas as pd
import numpy as np
from sklearn.preprocessing import LabelEncoder
import random
from sklearn.ensemble import RandomForestClassifier
from sklearn.ensemble import GradientBoostingClassifier
import matplotlib
import matplotlib.pyplot as plt
import sklearn
from scipy import stats
from sklearn.cluster import KMeans
import seaborn as sns
# Using Skicit-learn to split data into training and testing sets
from sklearn.model_selection import train_test_split
# Import the model we are using
from sklearn.ensemble import RandomForestRegressor
import os
os.environ["PATH"] += os.pathsep + 'C:/Program Files (x86)/Graphviz2.38/bin/'

features = pd.read_csv('prueba2.csv',sep=';')
print (features.head(5))

# Labels are the values we want to predict
labels = np.array(features['target'])
# Remove the labels from the features
# axis 1 refers to the columns
features= features.drop('target', axis = 1)
# Saving feature names for later use
feature_list = list(features.columns)
# Convert to numpy array
features = np.array(features)

# Split the data into training and testing sets
train_features, test_features, train_labels, test_labels = train_test_split(features, labels, test_size = 0.25, random_state = 42)

baseline_preds = test_features[:, feature_list.index('Violent crime')]
# Baseline errors, and display average baseline error
baseline_errors = abs(baseline_preds - test_labels)
print('Error: ', round(np.mean(baseline_errors), 2))

# Instantiate model with 1000 decision trees
rf = RandomForestRegressor(n_estimators = 1000, random_state = 42)
# Train the model on training data
rf.fit(train_features, train_labels);

# Use the forest's predict method on the test data
predictions = rf.predict(test_features)
# Calculate the absolute errors
errors = abs(predictions - test_labels)
# Print out the mean absolute error (mae)
print('Promedio del error absoluto:', round(np.mean(errors), 2), ' Porcentaje.')

# Calculate mean absolute percentage error (MAPE)
mape = 100 * (errors / test_labels)
# Calculate and display accuracy
accuracy = 100 - np.mean(mape)
print('Precision:', round(accuracy, 2), '%.')

# Get numerical feature importances
importances = list(rf.feature_importances_)
# List of tuples with variable and importance
feature_importances = [(feature, round(importance, 2)) for feature, importance in zip(feature_list, importances)]
# Sort the feature importances by most important first
feature_importances = sorted(feature_importances, key = lambda x: x[1], reverse = True)
# Print out the feature and importances 
[print('Variable: {:20} Importance: {}'.format(*pair)) for pair in feature_importances];

# Import tools needed for visualization
from sklearn.tree import export_graphviz
import pydot
# Pull out one tree from the forest
tree = rf.estimators_[5]
# Import tools needed for visualization
from sklearn.tree import export_graphviz
import pydot
# Pull out one tree from the forest
tree = rf.estimators_[5]
# Export the image to a dot file
export_graphviz(tree, out_file = 'tree.dot', feature_names = feature_list, rounded = True, precision = 1)
# Use dot file to create a graph
(graph, ) = pydot.graph_from_dot_file('tree.dot')
# Write graph to a png file

So my question is: how can I add a Confusion matrix to measure accuracy? I tried this example from this here, but it doesn't work. The following error appears:

enter image description here

Any advice?


From the code and task as your present it, a confusion matrix wouldn't make sense. This is because it shows how well a model is classifying samples i.e. saying which category they belong to. Your problem (as the author in your link states) is a regression problem, because you are predicting a continuous variable (temperature). Have a look here for more information.

In general, if you do have a classification task, printing the confusion matrix is a simple as using the sklearn.metrics.confusion_matrix function.

As input it takes your predictions and the correct values:

from sklearn.metrics import confusion_matrix

conf_mat = confusion_matrix(labels, predictions)

You could consider altering your task to make it be a classification problem, for example by grouping the temperatures in to classes of a given range.

You could say transform the target temperature to be a new_target_class, then change your code to use the [RandomForestClassifier][3].

I have done a quick and dirty conversion on the same data linked in that article, check it out here. I basically use the minimum and maximum values of the target variable to set a range, then aim for 10 different classes of temperature and create a new column in the table which assign that class to each row. The top it looks like this (click on picture to enlarge):

example of converting continuous to discrete target variable

If you can get those predictions going using the RandomForestClassifier, you can then run the confusion matrix code above on the results.

| improve this answer | |

Adding to @n1k31t4 's answer, you'll need to check the Confusion Matrix visually as heatmaps, especially while working on multiclass classification tasks:

# Visualise classical Confusion M0atrix
from sklearn.metrics import confusion_matrix
CM = confusion_matrix(labels, predictions)

# Visualize it as a heatmap
import seaborn

Heatmaps are super useful for very comlicated Confusion Matrices. They let you visualize data better than a simple table of numbers.

Below is an example found here:

enter image description here

| improve this answer | |

You can also use the sklearn built-in plot_confusion_matrix to plot.

from sklearn.metrics import plot_confusion_matrix
from sklearn.metrics import confusion_matrix, plot_confusion_matrix

conf_mat = confusion_matrix(labels, predictions, normalize='true')

disp = plot_confusion_matrix(classifier, X_test, y_test,
display_labels=class_names, cmap=plt.cm.Blues, normalize='true')

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.