I have a dataframe with a feature selection problem. I want to get the variables explaining the variance within each segment of the following dataset:
Do you agree Gender Age ... City Urban/Rural Output
0 Yes Female 25-34 ... Madrid Urban Will buy
1 No Male 18-25 ... Fès-Meknès Rural Won't
2 ... ... ... ... ... ... Undecided
....
The target being Output
.
I've been told a decision tree could be a way so after googling a bit I did:
# Feature Importance with Extra Trees Classifier
from sklearn.ensemble import ExtraTreesClassifier
# feature extraction
model = ExtraTreesClassifier(n_estimators=10)
model.fit(X, y)
print(model.feature_importances_)
Which returns:
[0. 0.00473011 0.00716472 0.00778101 0.0051573 0.00139121
0.02045262 0.00791912 0. 0.00222593 0.00173901 0.00417362
0.00222593 0.00313295 0.00565095 0.00652543 0. 0.00527774
0. 0. 0.00601354 0. 0. 0. ...
This looks exactly what I am looking for, but I don't get how we obtained this output. It look a bit like a black box. I am eager to learn and to do it myself to understand.Can you explain to me or refer me some resoures that would help me implement it myself?
Annex: data preparation
def load_dataset():
connection = psycopg2.connect(user = "user",
password = "password",
host = "host",
port = "5432",
database = "database")
connection.set_client_encoding('UTF8')
# connection.set_client_encoding('UNICODE')
sql = "select * from capi limit 10;"
# load the table
df = pd.read_sql_query(sql, connection)
# retrieve numpy array
dataset = df.values
# split into input (X) and output (y) variables
filtered_cols = ['Output']
cols = [col for col in cols if col not in filtered_cols]
X = df.loc[:, cols] #independent columns
X = X.astype(str)
y = df['Output'] #target column i.e price range
return X.values, y.values
X,y = load_dataset()