i have a dataset with two columns user posts (posts) and personality type (type) , i need personality type according to posts using this dataset so i used random forest regression for prediction here is my code:-

df = pd.read_csv('personality_types.csv')

count_vectorizer = CountVectorizer(decode_error='ignore')
X = count_vectorizer.fit_transform(df['posts'])
y = df['type'].values

Xtrain, Xtest, Ytrain, Ytest = train_test_split(X, y, test_size=0.33)

random_forest = RandomForestClassifier(n_estimators=100)
random_forest.fit(Xtrain, Ytrain)
Y_prediction = random_forest.predict(Xtest)


random_forest.score(Xtrain, Ytrain)
acc_random_forest = round(random_forest.score(Xtrain, Ytrain) * 100, 2)
print(round(acc_random_forest,2,), "%")


now i want to get prediction from a custom text how can i achive that ? how can i get personality type of a post separately using this model.


closed as unclear what you're asking by Mark.F, Toros91, Sean Owen Jan 14 at 19:00

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.


Use the CountVectorizer you have fitted to preprocess your custom input then feed it to your model for prediction.

custom_input = ['insert text here']
custom_input = count_vectorizer.transform(custom_input)
custom_prediction = random_forest.predict(custom_input)

Notice that you are getting a score on the train set, rather then on the test set (as you should)

random_forest.score(Xtrain, Ytrain)    # remove
random_forest.score(Xtest, Ytest)    # add
acc_random_forest = round(random_forest.score(Xtrain, Ytrain) * 100, 2)

Always be skeptic when getting 100% accuracy.


Not the answer you're looking for? Browse other questions tagged or ask your own question.