# Classification with frequency feature vector produces poor results

I'm classifying data into 2 classes, by Logistic Regression from python scikit-learn.

I'm trying different types of feature vectors:

• Binary
• Frequency
• TF-IDF

In Binary and TF-IDF feature vectors, I get great results. However, in Frequency feature vectors, I get poor results unless I multiply the values by some factor (for example 10, or 100). The multiplication improves the results significantly (the higher the factor, the better).

Is this a normal behavior or is it more likely that I'm doing something wrong? I suspected that the values are too small and my array rounded some of them down to zero because its type was merely float (unspecified bits number). But I tried changing it to np.float64, and the results didn't change.

What other factors can be causing this?

Edit: Learning Curves:

Binary | TF-IDF | Frequency X 10 (from left to right):

Normal frequency (3 attempts):

Edit 2: (data and code)

Frequency Data: X axis: a dictionary of 1K words , Y axis: 1K samples - 500 each class

Labels (1K binary labels):
note that it's a transpose vector. so X here is the samples

Code for frequency:

#loop...
vec[i] = np.divide(sample_vals, len(sample.split()))


Code for frequency X 10:

#loop...
vec[i] = np.multiply(np.divide(sample_vals, len(sample.split())), 10)


Classification code:

#in X I have the data, and in y I have the labels
scores = [clf.fit(X[train], y[train]).score(X[test], y[test])
for train, test in kfold.split(X)]

• Can you show all your learning curves? I suspect it is just a question of optimization (step size, iterations). Welcome to the site!
– Emre
May 19 '17 at 20:25
• @Emre Thank you! I added learning curves. Please take a look May 20 '17 at 16:59
• Your small frequency plots have not converged. What happens after a few thousand iterations? Are you persisting the loss only once every 200 iterations; why is it piecewise linear?
– Emre
May 20 '17 at 18:09
• Do you mean if I add more training examples? If so then I get this plot May 20 '17 at 18:13
• I'm not sure what you mean by persisting the loss. But I think you mean why do I have results only every 200 examples on the plot. I'm using this function to draw. Maybe I need to play with the parameters... May 20 '17 at 18:18

tfidf_vec = TfidfVectorizer(vocabulary=dictionaryList, use_idf=False)

use_idf=False is the solution. It lets it calculate frequency automatically. I don't have to manually divide by the sample's length this way.