31

Taken from this post:https://stats.stackexchange.com/a/245452/154812 The issue There are some issues with learning the word vectors using an "standard" neural network. In this way, the word vectors are learned while the network learns to predict the next word given a window of words (the input of the network). Predicting the next word is like predicting ...


25

You are running into that error because your X and Y don't have the same length (which is what train_test_split requires), i.e., X.shape[0] != Y.shape[0]. Given your current code: >>> X.shape (1, 6, 29) >>> Y.shape (29,) To fix this error: Remove the extra list from inside of np.array() when defining X or remove the extra dimension ...


19

I think in the original paper they suggest using $\log_2(N +1$), but either way the idea is the following: The number of randomly selected features can influence the generalization error in two ways: selecting many features increases the strength of the individual trees whereas reducing the number of features leads to a lower correlation among the trees ...


14

If the number of values belonging to each class are unbalanced, using stratified sampling is a good thing. You are basically asking the model to take the training and test set such that the class proportion is same as of the whole dataset, which is the right thing to do. If your classes are balanced then a shuffle (no stratification needed here) can ...


12

You should always do your evaluation of model performance on data that has not been over/undersampled. You can setup a pipeline with scikit-learn to perform your undersampling on the training set and then evaluate on the non-undersampled fold of data for each iteration as you described.


11

Great question... Here are some specific answers to your numbered questions: 1) You should cross validate on B not B`. Otherwise, you won't know how well your class balancing is working. It couldn't hurt to cross validate on both B and B` and will be useful based on the answer to 4 below. 2) You should test on both C and C` based on 4 below. 3) I would ...


9

I think subsampling (downsampling) is a popular method to control class imbalance at the base level, meaning it fixes the root of the problem. So for all of your examples, randomly selecting 1,000 of the majority of the class each time would work. You could even play around with making 10 models (10 folds of 1,000 majority vs the 1,000 minority) so you ...


8

If you have an adequate number of samples and want to use all the data, then k-fold cross-validation is the way to go. Having ~1,500 seems like a lot but whether it is adequate for k-fold cross-validation also depends on the dimensionality of the data (number of attributes and number of attribute values). For example, if each observation has 100 attributes, ...


8

You need to deal with imbalanced data set when the value of finding the minority class is much higher than that of finding the majority. Let say that 1% of the population have that rare disease. Suppose that you assign the same cost to saying that a healthy man is sick or saying that a sick man is healthy. Provide a model that say that everybody are healthy,...


7

Without making any underlying assumptions you will not get anywhere. That said, there are multi-arm bandit strategies that try to optimize the rewards, there is a ton of research on this field. It comes down to sampling from a distribution of your options (in your case two) and adapting this distribution based on the rewards. https://en.wikipedia.org/wiki/...


6

In Gradient Boosting the simple tree is built for only a randomly selected sub-sample of the full data set (random without replacement). While on the other hand, Random Forest the samples for each decision tree are selected via bootstrapping; sampling a dataset with replacement. Particularly for xgboost (see paper here)the ratio of sampling of each tree can ...


5

For 1) and 2), you want to 1) choose a model that performs well on data distributed as you expect the real data will be 2) evaluate the model on data distributed the same way So for those datasets, you shouldn't need to balance the classes. You might also try using class weights instead of under/oversampling, as this takes care of this decision for ...


5

This is a tough question to answer without more information. I'm going to assume that this is for model building, but without more detail it's hard to recommend something. However, there are some things which should generally be known: Population size How large is the population? Does your 2TB of data comprise the total population, or is this a sample of ...


5

I guess you say that you want to use 3-fold cross-validation because you know something about your data (that using k=10 would cause overfitting? I'm curious to your reasoning). I am not sure that you know this, if not then you can simply use a larger k. If you still think that you cannot use standard k-fold cross-validation, then you could modify the ...


5

You should use the testing set without any change, as answered by others. But it is very important to understand the difference between average accuracy and overall accuracy. In overall accuracy you find ( number of samples predicted correctly/ total number of samples) in average accuracy, you find the overall accuracy per class and then you find the ...


4

Yes, you can do all this using the Caret (http://caret.r-forge.r-project.org/training.html) package in R. For example, fitControl <- trainControl(## 10-fold CV method = "repeatedcv", number = 10, ## repeated ten times repeats = 10) gbmFit1 <- ...


4

It looks like you can sample the von Mises-Fisher distribution with that R package. Have you thought about calling R from within Python using the rpy2 package? I haven't tried this for myself, but could you try the following? from numpy import * import scipy as sp from pandas import * from rpy2.robjects.packages import importr import rpy2.robjects as ro ...


4

The accuracy is different because there are k-classifiers made for each number of k-folds, and a new accuracy is found. You don't select a fold yourself. K-Fold cross-validation is used to test the general accuracy of your model based on how you setup the parameters and hyper-parameters of your model fitting function. What you do select is the number of ...


4

OOB samples are a very efficient way to obtain error estimates for random forests. From a computational perspective, OOB are definitely preferred over CV. Also, it holds that if the number of bootstrap samples is large enough, CV and OOB samples will produce the same (or very similar) error estimates. Thus, if you perform many bootstrap samples, I would ...


4

Short answer: you need to deal with class imbalance if/because it makes your model better (on unseen data). "Better" is something that you have to define yourself. It could be accuracy, it could be a cost, it could be the true positive rate etc. Long answer: There is a subtle nuance that is important to grasp when talking about class imbalance. Namely, is ...


4

Ok, I just flipped the arguments in the loss: model.compile(loss=lambda loss, y_true: loss, optimizer='Adam') should be model.compile(loss=lambda y_true, loss: loss, optimizer='Adam') Enjoy


4

This is expected and is not related to SMOTE sampling. The computational complexity of non-linear SVM is on the order of $O(n^2)$ to $O(n^3)$ where $n$ is the number of samples. This means that if it takes 0.8 seconds for 7.5K data points, it should take [3, 48] minutes for 115K, $$[(115/7.5)^{2} \times 0.8, (115/7.5)^{3} \times 0.8]s=[3,48]m,$$and from 16 ...


4

I think SMOGN will work for your problem. The method is described in a paper titled: "SMOGN: a Pre-processing Approach for Imbalanced Regression". You can find it on arXiv. There is also a python implementation called "SMOGN" which can be installed through PyPI. You can find the package description at https://pypi.org/project/smogn/


3

Honestly there is no intuitive way to understand why NCE loss will work without deeply understanding its math. To understand the math, you should read the original paper. The reason why NCE loss will work is because NCE approximates maximum likelihood estimation (MLE) when the ratio of noise to real data $k$ increases. The TensorFlow implementation works ...


3

Data imbalance problem ?? In theory, it is only about numbers. Even if the difference is 1 sample it is data imbalance In practical, saying this is a data imbalance problem is controlled by three things: 1. The number and distribution of Samples you have 2. The variation within the same class 3. The similarities between different classes The last two ...


3

Imbalance is not defined formally but a ratio of 1 to 10 is usually imbalanced enough to benefit from using balancing technique. There are two type of imbalance, relative and absolute. In the relative the ratios between the majority and minority classes are imbalanced. In absolute you also have a small number of minority samples. The higher the imbalance ...


3

If you are willing to use the caret package in R and use random forests, you can use the method in the following blog post for downsampling with unbalanced datasets: http://appliedpredictivemodeling.com/blog/2013/12/8/28rmc2lv96h8fw8700zm4nl50busep Basically, you just add a single line to your train call. Here is the relevant part: > rfDownsampled <-...


3

My link in the comment has useful advice. I'd like to emphasis: This is a very well known fact in statistics Large sample size is good. There's nothing wrong with more and better quality data. It's silly to split up the data in your second point. You are voluntarily giving away information and statistical power. People do that for training an unbiased model,...


3

Isn't train_test_split expecting both X and Y to be a list of same length? Your X has length of 6 and Y has length of 29. May be try converting that to pandas dataframe (with 29x6 dimension) and try again? Given your data, it looks like you have 6 features. In that case, try to convert your X to have 29 rows and 6 columns. Then pass that dataframe to ...


3

Passing directly the output of the softmax is also common (among the few textual GANs out there), e.g. see the improved Wasserstein GANs (WGAN-GP). With hard Gumbel-softmax (+ straight-through estimator), you pass one-hot encoded vectors, which is the same as what you have with real data. If you pass the output of the softmax, the discriminator should be ...


Only top voted, non community-wiki answers of a minimum length are eligible