# improving accuracy of classification

I have data with 95 numeric variables and 5 categorical variables. My Y has 2 values. I built a decision tree and my accuracy was 81.8%. Then I created 3 new variables as follows. It improved accuracy to 84.3%

1. Normalize numeric variables and for training data, find mean vector for Y=1 and Y=0
2. for each data point, find euclidean distance from each mean vector - distance0 and distance1
3. third variable will be 0 if distance0 is <= distance1

I was wondering if there is any other new variables that i could create to improve the accuracy

I used a decision tree as it is fast to build and gives me indication whether a newly created variable is useful or not.

Please let me know if you have any thoughts

• Just a thought.. Can you apply PCA to your fields.. generate new fields and then run classifier (RF or XGBoost) and check the results. – Sandeep Bhutani Mar 21 '19 at 18:36

You could cluster your data with something like k-means and use the assigned cluster as a new feature.

import numpy as np
import pandas as pd
from sklearn.cluster import KMeans
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split

X = pd.DataFrame(data['data'], columns=data['feature_names'])
y = data['target']

X_train, X_test, y_train, y_test = train_test_split(X.copy(), y, test_size=0.2)

km = KMeans(n_clusters=3)
km.fit(X_train)
training_dist = km.transform(X_train)

X_train['dist1'] = training_dist[:,0]
X_train['dist2'] = training_dist[:,1]
X_train['dist3'] = training_dist[:,2]

clf = DecisionTreeClassifier()
clf.fit(X_train, y_train)

# new data
test_dist = km.transform(X_test)

X_test['dist1'] = test_dist[:,0]
X_test['dist2'] = test_dist[:,1]
X_test['dist3'] = test_dist[:,2]

clf.predict(X_test)

• How would you use this with new data that you have to predict? – G5W Mar 20 '19 at 15:13
• I did an example with distances – Simon Larsson Mar 20 '19 at 16:41