Timeline for Feature classification - am I doing it right?
Current License: CC BY-SA 4.0
12 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Mar 1, 2022 at 16:30 | comment | added | Erwan | @baltiturg If you have some external way to assign a weight to specific words, you can try. Otherwise in cases where the data is small it's preferable to make the features as simple as possible, in order to avoid overfitting. | |
| Feb 28, 2022 at 21:51 | comment | added | baltiturg | my big problem is that i have to analyze short sentences of <10 words, and decide what action i take per sentence, so, the frequency of a word means nothing because they all will appear one time probably, are there other ways to make this numeric and not binary in this case? thanks btw! | |
| Feb 28, 2022 at 21:28 | comment | added | Erwan | Your last point is about representing the importance of a word with a numerical value instead of only its existence with a boolean value. The most common options are to use the frequency of the word or its TFIDF weight, which is a combination of frequency with "inverse document frequency" which represents the importance of the word in the whole training data, assuming that rare words are more important. It's common but it doesn't always work better than boolean in practice, because it usually requires a higher volume of training data. | |
| Feb 28, 2022 at 21:25 | comment | added | Erwan | @baltiturg you're right that the order itself has no meaning. What I mean (and what matters) is that the order must be consistent: index i represents the same word in the training set and in the test set. However yes, you can calculate a distance between two vectors, but only distances which compare the same index in the two vectors. For boolean vectors there is the overlap coefficient, for numerical vectors cosine or Jaccard are the most common. Of course the location of the word is not taken into account (hence the term "bag of words"). | |
| Feb 28, 2022 at 21:18 | comment | added | baltiturg | also - what if when a feature exist i will rate it with another number, not 1 or 0, but lets say some features will have when exist 10, some will have 1, ( 10 when they are more important), how the model will behave? how do we choose the rating? other than binary 1/0 are there other way? hard to get info about this. | |
| Feb 28, 2022 at 20:55 | vote | accept | baltiturg | ||
| Feb 28, 2022 at 20:54 | comment | added | baltiturg | thanks again, when you say "the order must not be arbitrary", what you mean? those features are words, they become columns, how do you organize the columns initially? you have no way to organize the columns order other than just randomly becaues the order have no meaning. You mean probably that i index them once randomly then reuse them. But hey if I expect a vector like [1100110] and i get instead [0100110] it might seems they are closed, but they are not! because the first scalar could as well be in a different location. You can't measure vectors distance without order, no? | |
| Feb 28, 2022 at 17:48 | comment | added | Erwan | ... This can give better results but it's a more advanced level, I'd suggest you understand and try the traditional approach before attempting it. | |
| Feb 28, 2022 at 17:48 | comment | added | Erwan | Second point: the learning is what happens when the model finds the statistically significant patterns in the training data, building conditions like the one I gave as example. The goal of the learning is not for the model to understand what the words mean, but to discover which ones are associated with a particular label so that it can predict labels for fresh instances later on. Note that what you're describing about learning new words corresponds to some extent to what happens in deep learning: using word embeddings, the model can find semantic similarities between different words. ... | |
| Feb 28, 2022 at 17:42 | comment | added | Erwan |
@baltiturg as I said in the answer, the order must not be arbitrary, the vocabulary must be indexed once on the training data and then the indexes must be re-used. Otherwise the columns don't have any meaning indeed, so the classifier cannot do anything. In python you can use CountVectorizer: first you use fit_transform on the training set, then you use only transform on the test set.
|
|
| Feb 28, 2022 at 16:59 | comment | added | baltiturg | thanks a lot, but there are many things here that doesn't look good to me. First, the order i choose is arbitrary, which means if you basically just pick the closest vector to the input vector, you will be wrong, because i could flip this matrix as i wanted, the columns order had no real meaning hence the word "close" here doesn't really mean Euclidian close. Second, as you said i can not present new words/features, which means we are not really learning, just describing the vector dimension, so only if the input fit the EXACT vector we trained, we have a hit, otherwise, results are noise. | |
| Feb 28, 2022 at 16:43 | history | answered | Erwan | CC BY-SA 4.0 |