# Text classification- What to do when train and test data have different features

I am performing binary text classification. I have to classify a tweet 0 if neutral and 1 if hate speech.

So as general thumb rule i preprocessed my data. create term document frequency and After removing sparse terms i divide my data into train and test. I train my model using random forest and logistic regression and it worked fine.

set.seed(123)
tweetRand = randomForest(label ~ ., data = train_sparse, importance=TRUE, nTree=500 )
randPridct = predict(tweetRand, newdata = test_sparse)
table(test_sparse\$label,randPridct >=0.5)


Its is working fine on test data which divided from raw content. But when i am running it on a new unseen data it is throwing an exception.

> predicrRand_test=predict(tweetRand, newdata=sparse_4testing)
Error in eval(predvars, data, env) : object 'run' not found


My understanding is that 'run' is a feature present in training but not in unseen test data and during my model training 'run' was included in tdm. In preprocessing of test , run was not in test tdm.

SO how should i deal with these situation. I am new to data science. Please help.

My guess is that it is due to new levels of a categorical variable not present in the train data, but present in the unseen test data and this comes up as an error in the randomforest scoring. To take care of it you should select a randomized sample for test/train from your population and then harmonize the levels of the categorical variable using the techniques given here: