0
$\begingroup$

I have the columns in my Data Frame as shown below:

 Venue             city                Venue Categories

  Madison         London      [1, 1, 1, 1, 0, 0, 0, ...,0,0]
  WaterFront      Austria     [0, 1, 1 0, 0, 0,  0, ....0,1]
  Aeronaut        Marvilles   [0, 0, 0, 0, 1, 1, 1, ....1,1]
  Aeronaut        Paris       [0, 1, 1, 0, 0, 0, 0, ....1,1]
  Gostrich        New York    [0, 0, 1, 0, 0, 0, 0, ....1,0]

I am passing this data to my machine learning model , but model.fit is not accepting the input , My code is shown below , that I am trying ,

labelencoder = LabelEncoder()
dff['Venue']=labelencoder.fit_transform(dff['Venue'])
dff['city']=labelencoder.fit_transform(dff['city'])

Let's say , if I want to increase the number of features . I want to add more columns , then I again write everything for each feature just like shown below , if i want to add type column and owner column

city = dff['city'].values
owner = dff['owner'].values
type = dff['type'].values
categories = dff['Venue Categories'].values
labels = np.array(dff['Venue'].values)
data = np.array([make_sample(city[i], owner[i], type[i] categories[i]) for i in range(len(city))])

This will looks weird , I want to make it global , means there should not need to touch the code if we may increase the number of columns .

$\endgroup$
1
$\begingroup$

I managed to make it work, by combining the city column with the venue categories column into a 2D (numpy) array which can be used by the RandomForestClassifier of sklearn.

Example code:

import pandas as pd
from sklearn.preprocessing import LabelEncoder
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
import numpy as np

def make_data(df, target_column='Venue', categories_column='Venue Categories'):
    categories = df[categories_column].values
    other_features = []
    for col in df.columns:
        if col in [categories_column, target_column]: continue
        other_features.append(df[col].values);
    return np.array([[col[i] for col in other_features]+categories[i] for i in range(len(categories))]), np.array(df[target_column].values)

labelencoder = LabelEncoder()

dff['Venue'] = labelencoder.fit_transform(dff['Venue'])
dff['city'] = labelencoder.fit_transform(dff['city'])

data, labels = make_data(dff, 'Venue', 'Venue Categories')

train_data,test_data,train_labels,test_labels = train_test_split(data,labels,test_size=0.20)

model = RandomForestClassifier()
model.fit(train_data,train_labels)

Note make sure that Venue Categories column has same number of elements for each row of data, else a new problem will arise again. If needed fill with dummy values

$\endgroup$
12
  • $\begingroup$ ok will check it and get back to you $\endgroup$ – Nikos M. May 8 at 15:39
  • $\begingroup$ I think , it is the issue of some tensor , or something else , you may run the code on your side , if it works for you then share with me $\endgroup$ – Hamza May 8 at 16:55
  • $\begingroup$ see working example in updated answer. Managed to make it work by combining and converting city and venue categories into a 2D array suitable for RandomForestClassifier $\endgroup$ – Nikos M. May 8 at 19:28
  • $\begingroup$ Well , it is working , can you make it as a global . Means whenever any new column comes , it will auto convert into 2d array not by specifying every time for columns . $\endgroup$ – Hamza May 9 at 6:59
  • $\begingroup$ It is concatenating , means data has [4,1,0,1,..] . 4 specify city and next elements an array of categories . Don't you think that it should comes like [4,[1,0,1,1...]] . 4 should comes seperately $\endgroup$ – Hamza May 9 at 7:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.