New answers tagged supervised-learning
3
Scikit-learn has compose.ColumnTransformer which
allows different columns or column subsets of the input to be
transformed separately and the features generated by each transformer
will be concatenated to form a single feature space. This is useful
for heterogeneous or columnar data, to combine several feature
extraction mechanisms or transformations into a ...
4
Your model certainly overfits. It's likely that the main issue is the inclusion in the features of words which appear very rarely (especially those which appear only once in the corpus):
Words which appear only once don't help classification at all, if only because they can never be matched again. More generally, words which appear rarely are more likely to ...
1
To complete @BenjiAlbert answer, in case of imbalanced dataset, it is also recommended to use stratified k-fold to preserve the relative class frequencies in each fold. You can find more details in the sklearn user guide here.
4
If you're referring to the fact that your dataset is small:
You should use k-fold cross validation. This will let you evaluate your model on all 279 instances
If you're referring to the class imbalance being 31:202 in train and 8:48 in test:
Use AUROC and PRC to eliminate bias in thresholding
Also see MCC
1
I think in case of such unsymmetric data, where the output is outnumbered by one of the classes. Recall would be a good choice of measure than accuracy. The recall gives us the percentage of the relevant class actually predicted by the model.
1
In the article you provide, from page 11 results, I think one cannot conclude that transfer learning works better on smaller datasets than on larger ones.
If you look at the results of transfer learning score values (or RMSE) vs size of learning, it is also getting better while dataset size is increasing (for instance E2 or E5 or E8). So transfer learning ...
0
Transfer learning is in principle designed to utilise knowledge acquired from a training on a larger generic dataset (ie. animal pictures classification) to train a model that focuses on a more specific task using a smaller dataset (ie. cat breed pictures classification).
Transfer learning is otherwise called domain adaptation and in essence refers to the ...
Top 50 recent answers are included
