Skip to main content
Share Your Experience: Take the 2024 Developer Survey
44 votes
Accepted

train_test_split() error: Found input variables with inconsistent numbers of samples

You are running into that error because your X and Y don't have the same length (which is what ...
tuomastik's user avatar
  • 1,193
42 votes
Accepted

Intuitive explanation of Noise Contrastive Estimation (NCE) loss?

Taken from this post:https://stats.stackexchange.com/a/245452/154812 The issue There are some issues with learning the word vectors using a "standard" neural network. In this way, the word ...
user154812's user avatar
23 votes
Accepted

How many features to sample using Random Forests

I think in the original paper they suggest using $\log_2(N +1$), but either way the idea is the following: The number of randomly selected features can influence the generalization error in two ways: ...
oW_'s user avatar
  • 6,377
14 votes

Is stratified sampling necessary (random forest, Python)?

If the number of values belonging to each class are unbalanced, using stratified sampling is a good thing. You are basically asking the model to take the training and test set such that the class ...
Kiritee Gak's user avatar
  • 1,799
13 votes
Accepted

Cross validation for highly imbalanced data with undersampling

You should always do your evaluation of model performance on data that has not been over/undersampled. You can setup a pipeline with scikit-learn to perform your undersampling on the training set and ...
Wes's user avatar
  • 692
11 votes
Accepted

When should we consider a dataset as imbalanced?

I think subsampling (downsampling) is a popular method to control class imbalance at the base level, meaning it fixes the root of the problem. So for all of your examples, randomly selecting 1,000 of ...
TBSRounder's user avatar
8 votes
Accepted

Why do we need to handle data imbalance?

You need to deal with imbalanced data set when the value of finding the minority class is much higher than that of finding the majority. Let say that 1% of the population have that rare disease. ...
DaL's user avatar
  • 2,643
8 votes
Accepted

How are samples selected from training data in Xgboost

In Gradient Boosting the simple tree is built for only a randomly selected sub-sample of the full data set (random without replacement). While on the other hand, Random Forest the samples for each ...
Carlos Mougan's user avatar
7 votes

Imbalanced dataset: how to deal with test data?

You should use the testing set without any change, as answered by others. But it is very important to understand the difference between average accuracy and overall accuracy. In overall accuracy you ...
Bashar Haddad's user avatar
7 votes
Accepted

Exploration vs exploitation tradeoff to find a price that maximizes revenue

Without making any underlying assumptions you will not get anywhere. That said, there are multi-arm bandit strategies that try to optimize the rewards, there is a ton of research on this field. It ...
Jan van der Vegt's user avatar
7 votes
Accepted

SMOTE for regression

I think SMOGN will work for your problem. The method is described in a paper titled: "SMOGN: a Pre-processing Approach for Imbalanced Regression". You can find it on arXiv. There is also a python ...
Mojtaba's user avatar
  • 86
5 votes

Why do we need to handle data imbalance?

Short answer: you need to deal with class imbalance if/because it makes your model better (on unseen data). "Better" is something that you have to define yourself. It could be accuracy, it could be a ...
Valentin Calomme's user avatar
5 votes
Accepted

How to do k-folds in python whilst splitting into 3 sets?

I don't think there's a builtin way to do it, but the two methods you've mentioned combine pretty nicely to do the job: ...
Ben Reiniger's user avatar
  • 11.9k
5 votes
Accepted

Over-sampling: is my model over-fitting?

In order to get accurate results, you should not oversample the test set! Otherwise you are simply evaluating on synthetic samples that you yourself have created. The support on your classification ...
Djib2011's user avatar
  • 7,998
4 votes

When should we consider a dataset as imbalanced?

Imbalance is not defined formally but a ratio of 1 to 10 is usually imbalanced enough to benefit from using balancing technique. There are two type of imbalance, relative and absolute. In the ...
DaL's user avatar
  • 2,643
4 votes

Strategy for dealing with giant sample size

My link in the comment has useful advice. I'd like to emphasis: This is a very well known fact in statistics Large sample size is good. There's nothing wrong with more and better quality data. It's ...
SmallChess's user avatar
  • 3,560
4 votes
Accepted

Which is better: Out of Bag (OOB) or Cross-Validation (CV) error estimates?

OOB samples are a very efficient way to obtain error estimates for random forests. From a computational perspective, OOB are definitely preferred over CV. Also, it holds that if the number of ...
oW_'s user avatar
  • 6,377
4 votes

K-Fold Cross validation confusion?

The accuracy is different because there are k-classifiers made for each number of k-folds, and a new accuracy is found. You don't select a fold yourself. K-Fold cross-validation is used to test ...
Eric C. Bohn's user avatar
4 votes

Keras negative sampling with custom layer

Ok, I just flipped the arguments in the loss: model.compile(loss=lambda loss, y_true: loss, optimizer='Adam') should be ...
cadama's user avatar
  • 191
4 votes

Oversampling before Cross-Validation, is it a problem?

I suggest having a read of this article. The article explains: When upsampling before cross validation, you will be picking the most oversampled model, because the oversampling is allowing data to ...
sums22's user avatar
  • 427
4 votes

SVM SMOTE fit_resample() function runs forever with no result

This is expected and is not related to SMOTE sampling. The computational complexity of non-linear SVM is on the order of $O(n^2)$ to $O(n^3)$ where $n$ is the number of samples. This means that if it ...
Esmailian's user avatar
  • 9,322
4 votes
Accepted

Why did sampling boost the performance of my model?

As you mentioned in a comment, you are upsampling before splitting the test set, which leads to data leakage; your scores are not to be trusted. The problem is that a given positive sample may be ...
Ben Reiniger's user avatar
  • 11.9k
4 votes
Accepted

Is sampling a valid way to reduce complexity?

I would get a sufficiently large random/representative sample and cluster that. To see what is such a sample, you will have to get two such samples and cluster them to get cluster solutions c1 and c2. ...
kangaroo_cliff's user avatar
3 votes

train_test_split() error: Found input variables with inconsistent numbers of samples

Isn't train_test_split expecting both X and Y to be a list of same length? Your X has length of 6 and Y has length of 29. May be ...
Sal's user avatar
  • 276
3 votes

Sample selection through clustering

Clustering will be much too expensive for your purpose (most are O(n^2), and the good ones like HAC may even be O(n^3) - you won't be able to run them on 300k instances). Also beware of the ...
Has QUIT--Anony-Mousse's user avatar
3 votes

Intuitive explanation of Noise Contrastive Estimation (NCE) loss?

[I've added this answer as I think others miss the main theoretical gist.] Firstly, NCE and Negative Sampling (NS) serve different purposes: NS is a generic trick used to train a classifier if you ...
Carl's user avatar
  • 396
3 votes

When should we consider a dataset as imbalanced?

Data imbalance problem ?? In theory, it is only about numbers. Even if the difference is 1 sample it is data imbalance In practical, saying this is a data imbalance problem is controlled by three ...
Bashar Haddad's user avatar
3 votes

Intuitive explanation of Noise Contrastive Estimation (NCE) loss?

Honestly there is no intuitive way to understand why NCE loss will work without deeply understanding its math. To understand the math, you should read the original paper. The reason why NCE loss will ...
Lei Mao's user avatar
  • 49
3 votes
Accepted

Gumbel Softmax vs Vanilla Softmax for GAN training

Passing directly the output of the softmax is also common (among the few textual GANs out there), e.g. see the improved Wasserstein GANs (WGAN-GP). With hard Gumbel-softmax (+ straight-through ...
noe's user avatar
  • 26.9k
3 votes

Downsampling and class ratios

Downsampling means you sample from the majority class (the 98.5%) to reduce the imbalance between majority and minority class. If you keep the ratio constant you simply reduce your number of trainings ...
oW_'s user avatar
  • 6,377

Only top scored, non community-wiki answers of a minimum length are eligible