New answers tagged svm
1
You need to create a vocabulary of the n-grams, i.e., a numbered inventory of bigrams that you are going to use as features. Typically, these are the most frequent ones. When you create the feature vector, you start with a zero vector and put one (or add one) if the n-gram with the corresponding index appears is in your sentence.
Machine learning libraries ...
1
I will say, it's an either Or situation
You can pick one of "Incremental/Online" training Or "addition of new class".
You may do a fine-tuning approach with a Neural network by adjusting the o/p layer and training the last few layers. But this approach expects the new data to be quite similar to the training set.
KNN - Can do the online ...
2
Similarly to NB or kNN, the DT and SVM algorithms work with the features which are provided as input. So whenever ML is applied to text it's important to understand how the unstructured text is transformed into structured data, i.e. how text instances are represented with features.
There are many options, but traditionally a document is represented as as a ...
0
Please try duplicating the specific company's data ten times or more, and include more samples in cross/test data from that company-specific data (3:1). I hope this will have some positive implications.
0
I don't think that's a very good idea: the goal is not to make the model predict a more extreme polarity when the tweet relates to the company.
Instead you might want to consider oversampling the few instances of this specific company. For instance if you have 100 company-specific tweets and 1000 general tweets in your training set, you could duplicate the ...
0
I understand you are looking for some interpretability.
But if you recall Feature engineering, we mostly remove features which are of less value. What it means is that all the remaining features are contributing.
What you may do a trade-off between Accuracy and Interpretability -
Logistics Regression and Decision Tree will give you a clear picture on how ...
0
The sample_scores values, along with a cutoff threshold value, are used to determine whether a value is an outlier or not. You should be careful if you try to compare these sample_score values to see which values may be more anomalous than others. If you are looking to eliminate outliers solely to eliminate them from your data-set, using this value alone on ...
0
Old question, but in case anyone is still interested in an answer...
In the perceptron algorithm a point $x$ has a label $y$ equal to 1 or -1. The predicted label for a point is $w\cdot x$. The goal is to separate the points with an hyperplane orthogonal to w in such a way that the points with label 1 are on one side and the points with label -1 are on the ...
0
This error usually indicates your train and test in the train_test_split() function call are different sizes. You may need to reshape one to get them to match. Look at the shapes of train and test to see what is the problem.
1
Accuracy is not the best measure for imbalanced data. Prefer precision and recall.
Do undersampling/oversampling to get equal samples for each class and try XGBoost.
Or else you can use SVC with class weights, give lower class weight to classes with more samples and vice versa.
1
My answer would be to perform LDA in each fold of your cross-validation.
The reason is the following. Cross-validation is used as a way to get an estimate of the performance of the model. This estimate essentially attempts to answer the following question:
How will my model perform when trained on an arbitrary set of data
If you don't use cross-...
-1
Sometimes k-fold is not necessary to bring better results than the standard LDA. It is preferable for large data set.
1
It depends what you mean by "can be used": any regression algorithm can be used, the question is how reliably it would perform. You can compare different algorithms experimentally (if you have a dataset).
[Updated after question edited]
In general the way to use ML with this kind of setting is to train a classification model based only on the categorical ...
0
This collection of twitter datasets might help you find the dataset you're looking for. Mainly sentiment analysis datasets, but moderation and classification datasets too.
Top 50 recent answers are included
