# Tag Info

Accepted

### train_test_split() error: Found input variables with inconsistent numbers of samples

You are running into that error because your X and Y don't have the same length (which is what ...
• 1,153
Accepted

### Intuitive explanation of Noise Contrastive Estimation (NCE) loss?

Taken from this post:https://stats.stackexchange.com/a/245452/154812 The issue There are some issues with learning the word vectors using an "standard" neural network. In this way, the word vectors ...
• 514
Accepted

### How many features to sample using Random Forests

I think in the original paper they suggest using $\log_2(N +1$), but either way the idea is the following: The number of randomly selected features can influence the generalization error in two ways: ...
• 6,035

### Is stratified sampling necessary (random forest, Python)?

If the number of values belonging to each class are unbalanced, using stratified sampling is a good thing. You are basically asking the model to take the training and test set such that the class ...
• 1,769
Accepted

### Cross validation for highly imbalanced data with undersampling

You should always do your evaluation of model performance on data that has not been over/undersampled. You can setup a pipeline with scikit-learn to perform your undersampling on the training set and ...
• 672
Accepted

### With unbalanced class, do I have to use under sampling on my validation/testing datasets?

Great question... Here are some specific answers to your numbered questions: 1) You should cross validate on B not B`. Otherwise, you won't know how well your ...
• 6,708
Accepted

### When should we consider a dataset as imbalanced?

I think subsampling (downsampling) is a popular method to control class imbalance at the base level, meaning it fixes the root of the problem. So for all of your examples, randomly selecting 1,000 of ...
• 853
Accepted

### Why do we need to handle data imbalance?

You need to deal with imbalanced data set when the value of finding the minority class is much higher than that of finding the majority. Let say that 1% of the population have that rare disease. ...
• 2,593

### Imbalanced dataset: how to deal with test data?

You should use the testing set without any change, as answered by others. But it is very important to understand the difference between average accuracy and overall accuracy. In overall accuracy you ...
• 1,859
Accepted

### Exploration vs exploitation tradeoff to find a price that maximizes revenue

Without making any underlying assumptions you will not get anywhere. That said, there are multi-arm bandit strategies that try to optimize the rewards, there is a ton of research on this field. It ...
• 9,118
Accepted

### How are samples selected from training data in Xgboost

In Gradient Boosting the simple tree is built for only a randomly selected sub-sample of the full data set (random without replacement). While on the other hand, Random Forest the samples for each ...
• 5,621

### With unbalanced class, do I have to use under sampling on my validation/testing datasets?

For 1) and 2), you want to ...
• 3,037
Accepted

### What is a "good" sample size

This is a tough question to answer without more information. I'm going to assume that this is for model building, but without more detail it's hard to recommend something. However, there are some ...
Accepted

### SMOTE for regression

I think SMOGN will work for your problem. The method is described in a paper titled: "SMOGN: a Pre-processing Approach for Imbalanced Regression". You can find it on arXiv. There is also a python ...
• 66
Accepted

### Over-sampling: is my model over-fitting?

In order to get accurate results, you should not oversample the test set! Otherwise you are simply evaluating on synthetic samples that you yourself have created. The support on your classification ...
• 7,618
Accepted

### Avoid iterations while calculating average model accuracy

Yes, you can do all this using the Caret (http://caret.r-forge.r-project.org/training.html) package in R. For example, ...
• 923

### Sampling from a multivariate von Mises-Fisher distribution in Python

It looks like you can sample the von Mises-Fisher distribution with that R package. Have you thought about calling R from within Python using the rpy2 package? I haven't tried this for myself, but ...
• 426

### K-Fold Cross validation confusion?

The accuracy is different because there are k-classifiers made for each number of k-folds, and a new accuracy is found. You don't select a fold yourself. K-Fold cross-validation is used to test ...
• 141
Accepted

### Which is better: Out of Bag (OOB) or Cross-Validation (CV) error estimates?

OOB samples are a very efficient way to obtain error estimates for random forests. From a computational perspective, OOB are definitely preferred over CV. Also, it holds that if the number of ...
• 6,035

### Why do we need to handle data imbalance?

Short answer: you need to deal with class imbalance if/because it makes your model better (on unseen data). "Better" is something that you have to define yourself. It could be accuracy, it could be a ...
• 5,206

### Keras negative sampling with custom layer

Ok, I just flipped the arguments in the loss: model.compile(loss=lambda loss, y_true: loss, optimizer='Adam') should be ...
• 191
Accepted

### How to do k-folds in python whilst splitting into 3 sets?

I don't think there's a builtin way to do it, but the two methods you've mentioned combine pretty nicely to do the job: ...
• 10.2k

### SVM SMOTE fit_resample() function runs forever with no result

This is expected and is not related to SMOTE sampling. The computational complexity of non-linear SVM is on the order of $O(n^2)$ to $O(n^3)$ where $n$ is the number of samples. This means that if it ...
• 8,667
Accepted

### Why did sampling boost the performance of my model?

As you mentioned in a comment, you are upsampling before splitting the test set, which leads to data leakage; your scores are not to be trusted. The problem is that a given positive sample may be ...
• 10.2k
Accepted

### Is sampling a valid way to reduce complexity?

I would get a sufficiently large random/representative sample and cluster that. To see what is such a sample, you will have to get two such samples and cluster them to get cluster solutions c1 and c2. ...
• 362

### Sample selection through clustering

Clustering will be much too expensive for your purpose (most are O(n^2), and the good ones like HAC may even be O(n^3) - you won't be able to run them on 300k instances). Also beware of the ...

### Intuitive explanation of Noise Contrastive Estimation (NCE) loss?

Honestly there is no intuitive way to understand why NCE loss will work without deeply understanding its math. To understand the math, you should read the original paper. The reason why NCE loss will ...
• 39

### Intuitive explanation of Noise Contrastive Estimation (NCE) loss?

[I've added this answer as I think others miss the main theoretical gist.] Firstly, NCE and Negative Sampling (NS) serve different purposes: NS is a generic trick used to train a classifier if you ...
• 336
Accepted

### Poor performance shown on Rare event modeling

If you are willing to use the caret package in R and use random forests, you can use the method in the following blog post for downsampling with unbalanced datasets: http://appliedpredictivemodeling....
• 426