I have data with 95 numeric variables and 5 categorical variables. My Y has 2 values. I built a decision tree and my accuracy was 81.8%. Then I created 3 new variables as follows. It improved accuracy to 84.3%

  1. Normalize numeric variables and for training data, find mean vector for Y=1 and Y=0
  2. for each data point, find euclidean distance from each mean vector - distance0 and distance1
  3. third variable will be 0 if distance0 is <= distance1

I was wondering if there is any other new variables that i could create to improve the accuracy

I used a decision tree as it is fast to build and gives me indication whether a newly created variable is useful or not.

Please let me know if you have any thoughts

  • 1
    $\begingroup$ Just a thought.. Can you apply PCA to your fields.. generate new fields and then run classifier (RF or XGBoost) and check the results. $\endgroup$ – Sandeep Bhutani Mar 21 '19 at 18:36

You could cluster your data with something like k-means and use the assigned cluster as a new feature.

import numpy as np
import pandas as pd
from sklearn.datasets import load_iris
from sklearn.cluster import KMeans
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split

data = load_iris()

X = pd.DataFrame(data['data'], columns=data['feature_names'])
y = data['target']

X_train, X_test, y_train, y_test = train_test_split(X.copy(), y, test_size=0.2)

km = KMeans(n_clusters=3)
training_dist = km.transform(X_train)

X_train['dist1'] = training_dist[:,0]
X_train['dist2'] = training_dist[:,1]
X_train['dist3'] = training_dist[:,2]

clf = DecisionTreeClassifier()
clf.fit(X_train, y_train)

# new data
test_dist = km.transform(X_test)

X_test['dist1'] = test_dist[:,0]
X_test['dist2'] = test_dist[:,1]
X_test['dist3'] = test_dist[:,2]

  • $\begingroup$ How would you use this with new data that you have to predict? $\endgroup$ – G5W Mar 20 '19 at 15:13
  • $\begingroup$ I did an example with distances $\endgroup$ – Simon Larsson Mar 20 '19 at 16:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.