0
$\begingroup$

I am getting value error while trying to classify using TfidfVectorizer. I have looked in the link below but couldn't able to find a solution.

[Binary text classification with TfidfVectorizer gives ValueError: setting an array element with a sequence

The code is as below. I know it is because of train_test_split but no clue. please help.

from sklearn.feature_extraction.text import TfidfVectorizer
tf_idf=TfidfVectorizer(stop_words='english',strip_accents='ascii',max_features=500)

tf_idf_matrix=tf_idf.fit_transform(data['Text'])

data_extra_features=pd.concat([data,pd.DataFrame(tf_idf_matrix.toarray(),columns=tf_idf.get_feature_names())],axis=1)
from sklearn.model_selection import train_test_split
X=data_extra_features
features=X.columns.drop(['Value','Text'])
target=['Value']
X_train,X_test,y_train,y_test=train_test_split(X[features],X[target])

#Decision Tree
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import accuracy_score


def model_accuray(model,X_train,X_test,y_train,y_test):

    model.fit(X_train,y_train)
    pred=dt.predict(X_test)
    print('Traning accurarcy',accuracy_score(y_train,dt.preditct(X_train)))
    print('Testing accurarcy',accuracy_score(y_test,pred))

from sklearn.naive_bayes import MultinomialNB
from sklearn.linear_model import LogisticRegression
#NaiveBayes model

mb=MultinomialNB()
lr=LogisticRegression()
dt=DecisionTreeClassifier(min_samples_split=40)

# model_accuray(mb,X_train,X_test,y_train,y_test)
# model_accuray(lr,X_train,X_test,y_train,y_test)
model_accuray(dt,X_train,X_test,y_train,y_test)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.