I'm trying to choose an algorithm for filtering spam. I found two options:
Create word dictionary for spam and not spam data. Calculate average TF-IDF for each word and use cosine similarity for filtering.
Or use word dictionary for training logistic regression model.
Could you suggest what fit the best for my goal. Maybe I should use some another algorithm.
Use logistic regression which allows the weights to be learnt. By using cosine similarity you are forcing the weights to be the same for all features (assuming that you normalize the features first). This is putting unnecessary restrictions on the model.
You have to make a prediction (after performing supervised learning), so cosine distance won't help since it's fundamentally used in unsupervised learning (clustering and distance). A popular spam prediction approach was published in Springer textbook by Hastie & Tibshirani, which is on statistical learning. You could probably find papers online on spam prediction.
Logistic Regression is great for CTR and spam filtering (text data in general) thanks to the use of the hashing trick. Vowpal Wabbit has an optimized implementation of it in case you have GB of data. I guess neural networks will perform well too in these kind of tasks.
https://github.com/JohnLangford/vowpal_wabbit
One recommendation that applies to every other data science problem; the biggest improvements you can get come from creating new features. In your case: TF-IDF, n-grams, POS Tagging can be used to improve the classifier more than just choosing A or B algorithm.
Note: Vowpal Wabbit can create n-grams and interactions automatically. Other types of pre-processing should be coded separately.
