Questions tagged [sampling]
The sampling tag has no usage guidance.
0
votes
1answer
24 views
Choosing sample from large dataset?
How to choose sample from a large dataset such that each unique row from the dataset is selected at least once in the sample? Is there a way of doing this in python?
1
vote
1answer
35 views
Using SMOTE for Synthetic Data generation to improve performance on unbalanced data
I presently have a dataset with 21392 samples, of which, 16948 belong to the majority class (class A) and the remaining 4444 belong to the minority class (class B). I am presently using SMOTE (...
0
votes
0answers
20 views
Cross-validation and out-of-bag bootstrap applications
I have a question regarding steps on which a specific resample method should be used in general.
As far as I know:
out-of-bag bootstrap is the resample method with replacement, which has lower ...
1
vote
1answer
26 views
2
votes
0answers
15 views
Sub-sampling so that sample statistics match population statistics
I want to investigate the impact of various testing strategies on a product. Let's say chairs. I start with 500 random chairs that I've picked up from garage/yard sales. They come in all shapes and ...
0
votes
0answers
28 views
Why am I getting such a low precision after performing oversampling and undersampling?
I am performing fraud analysis on credit card fraud committed dataset.
I am performing oversampling by .sample(oversampled_class_size) and undersampling by .sample(undersampled_class_size).
I am ...
2
votes
1answer
24 views
In Machine Learning, what is the point of using stratified sampling in selecting test set data?
I am currently learning machine learning via this book "Hands-On Machine Learning with Sci-kit learn and Tensorflow" by Aurelien Geron.
In page 76 and 77, the author talks about using stratified ...
5
votes
1answer
144 views
Cross validation for highly imbalanced data with undersampling
In my problem, I am dealing with a highly imbalanced data set, say for every positive class there are 10000 negative one. A normal starting method to train a model is to undersample the data. In this ...
0
votes
0answers
28 views
Relation between using stratify and class weights for imbalanced classes
I'm working on a multi-class classification problem where the classes are imbalanced (70:25:5).
Train-Test Split
...
0
votes
0answers
9 views
sample n unique items from dataset
I have a dataset that has N of different unique items and each item appears Ai times (every item appears different times).
This is mean that I have the ...
1
vote
1answer
68 views
Oversampling before Cross-Validation, is it a problem?
I have a multi-class classification problem to solve which is highly imbalanced. Obviously I'm doing oversampling, but I'm doing cross-validation with the over-sampled dataset, as a result of which I ...
0
votes
2answers
45 views
Disadvantages of hyperparameter tuning on a random sample of dataset
I often work with very large datasets where it would be impractical to check all relevant combinations of hyperparameters when constructing a machine learning model. I'm considering randomly sampling ...
0
votes
1answer
14 views
Optimal proportion between the amount of Class = 1 and the amount of Class = 0?
I am quite new machine learning methods, so I may not write proper technical formulas.
My question is about the optimal proportion between sample size in Class = 1 ...
-1
votes
0answers
28 views
Should the test set be undersampled in a way that mirrors the distribution of the training set?
I have a balanced dataset that I want to "force" an imbalance on. So I've removed some % of the instances of class A from the training set.
However, the test set is balanced. In order to get an ...
1
vote
0answers
12 views
SmoteBoost: Should SMOTE be ran individually for each iteration/tree in the boosting?
As per the paper on SmoteBoost, SMOTE is ran for each iteration of the boosting, generating N samples, which are further added to the original training data and the weight distribution of the ...
-1
votes
0answers
5 views
When is a weather forecast 'in-sample'?
I've got some weather forecast data and I want to split it into a sample for analysis (in-sample) and a sample for testing (out-of-sample), to avoid over-fitting to the data.
I made the choice to ...
0
votes
1answer
43 views
How can I extract bootstrap generated datasets into individual dataframes?
I am having a bit of trouble understanding Bootstrapping and what/how I can manipulate the bootstrap created dataset.
This is all in R
My original dataset is structured like this:
...
1
vote
1answer
84 views
A few questions to understand a random forest blog [closed]
I'm trying to understand a nice blog on the trade-off between sensitivity versus specificity with the random forest and logistic regression models. I have a few questions:
1) The blog used a 10 fold ...
0
votes
3answers
120 views
Downsampling and class ratios
My target variable is whether an application is accepted or not. It is a highly imbalanced target with 98.5% of applications accepted. I am unclear about the concept of downsampling. If I were to ...
0
votes
1answer
31 views
Generating a set of different scenarios based on some initial observations
I have a in my hands 3 different time series which model 3 different scenarios (base, downside, upside). Every of this time-series depends on a set of 11 different attributes, which take values for ...
1
vote
1answer
27 views
Difference between bagging and boosting
Can anyone explain me the basic difference between bagging and boosting and which technique can be used in which scenario?
2
votes
1answer
819 views
How to correctly perform data sampling for train/test split in multi-label dataset?
Problem statement
I have a text multi-label classification dataset, and I've found a problem with the dataset sampling.
I'm facing two different strategies. The first one consists in preprocessing ...
0
votes
1answer
78 views
Downsampling the dataset to create balanced dataset for neural models
I have a classification dataset with 10k instances and 4 classes and it is unbalanced. 7000 of it belongs to first class, 2000 of it belongs to second 800 of it belongs to third class and remaining ...
1
vote
0answers
107 views
Sample size equation for multi-class distribution
I have a large (k>15) number of potential classes involved in a text classification problem, and don't know the true distribution of these classes in the ...
1
vote
0answers
20 views
Downsample GPS track
I am working with GPS track files (list of X and Y coordinates). I have tracks with a high sampling rate and want to downsample the track for easier handling.
The obvious way would be to create a new ...
1
vote
1answer
42 views
I have limited samples for one class, unlimited samples for the other class. Need to balance?
I want my machine learning algorithm to learn the difference between two classes, actually picture of X or ...
2
votes
0answers
122 views
Oversampling for multi-class neural net
Does this make sense or do I have no idea what I'm doing?
I want to train a model that takes a sentence and outputs a binary multi-class vector of size $K$ where each dimension is a question class.
...
2
votes
1answer
290 views
Gumbel Softmax vs Vanilla Softmax for GAN training
When training a GAN for text generation, i have seen many people feeding the gumbel-softmax from the generator output and feed into the discriminator. This is to bypass the problem of having to sample ...
4
votes
1answer
682 views
Why is sampling useful in machine learning?
I have met that question online and I wanted to know where sampling can simulate complex processes and why?
Why is sampling useful in machine learning?
Sampling can increase the accuracy of the ...
2
votes
1answer
48 views
Overfitted model produces similar AUC on test set, so which model do I go with?
I was trying to compare the effect of running GridSearchCV on a dataset which was oversampled prior and oversampled after the training folds are selected. The oversampling approach I used was random ...
0
votes
1answer
38 views
Generating ordinal data
I would like to generate synthetic data which are ordinal, i.e. ordered, in Python. But how would I do this? What are the differences in generating ordinal data vs categorical data?
I'm reading the ...
1
vote
0answers
24 views
Search Query Sample Size Determination for validation set
While designing a search system, which searches in N identifiable categories, how many search queries does one need in each category to validate the target metric (DCG) scores accurately (balanced ...
0
votes
1answer
37 views
R programming (Jackknife) [closed]
Hi I would like to ask how to sample out 50 instances from 150 instances of iris data by using Jackknife.
Is it possible?
Thanks in advance
1
vote
1answer
97 views
Right ML mode and metric to minimize FN and FP on imbalanced dataset
So I have a dataset in which I have to predict class binary label (1 or 0), the problem, out of 120k data points, only 200 have the label '1'.
the aim is to minimize FN and FP.
Which ML model should ...
1
vote
0answers
96 views
Orange: Group samples by a “splitting” feature for cross-validation?
I need to split my datasets based on my own feature column in order to hold together certain data rows (e.g. from one patient, or compound) for cross-validation (CV).
In Orange 3.11, Test&Score ...
1
vote
2answers
206 views
Working with audio data with different sample rates in Tensorflow
I am trying to implement (as a toy project) some aspects of speech recognition in Tensorflow. The audio files I want to use as training and test data have different sample rates (16, 20, 44 and 44.1 ...
-1
votes
0answers
128 views
RBM, Gibbs Sampling, and Real-Valued Data
I am admittedly very new to Restricted Boltzmann Machines (RBM) and have been toying with the idea of using RBM to generate samples from the underlying distribution of ...
2
votes
1answer
2k views
Keras negative sampling with custom layer
I am trying to implement negative sampling in Keras. I wrote the following code that just compute the loss and I plan to add an additional output for the logits once I get it up and running. Here is ...
4
votes
1answer
104 views
Exploration vs exploitation tradeoff to find a price that maximizes revenue
Is there a practical strategy that can learn to price a product optimally? Right now I have the following arbitrary hill-climbing algorithm:
Run an experiment at starting price ...
3
votes
2answers
13k views
SMOTE and multi class oversampling
I have read that the SMOTE package is implemented for binary classification. In the case of n classes, it creates additional examples for the smallest class. Can I balance all the classes by running ...
2
votes
1answer
44 views
Stratified Sampling Variable Choice
I am trying to do stratified sampling in R to sample from my data and one of the parameters is group, which takes variable names to sample from keeping same initial distribution of the data set.
Is ...
9
votes
2answers
1k views
why we need to handle data imbalance?
I need to know why we need to deal with data imbalance. I know how to deal with it and different methods to solve the issue which is by up sampling or down sampling or by using Smote.
For example, if ...
8
votes
1answer
4k views
How many features to sample using Random Forests
The Wikipedia page which quotes "The Elements of Statistical Learning" says:
Typically, for a classification problem with $p$ features, $\lfloor \sqrt{p}\rfloor$ features are used in each split.
...
1
vote
0answers
25 views
Resampling a normally distributed dataset for regression problems?
I have a dataset from an operating process having 5 measurements and 1 outcome. All values are normally distributed. When I train a regression model on the dataset it performs good on the majority of ...
2
votes
1answer
1k views
Which is better: Out of Bag (OOB) or Cross-Validation (CV) error estimates?
I have seen other posts in this forum but didn't find any convincing answer.
Random Forest has an another way of tuning hyperparameter via OOB by design. OOB and CV are not the same as OOB error is ...
12
votes
2answers
36k views
train_test_split() error: Found input variables with inconsistent numbers of samples
Fairly new to Python but building out my first RF model based on some classification data. I've converted all of the labels into int64 numerical data and loaded into X and Y as a numpy array, but I am ...
0
votes
1answer
3k views
K-Fold Cross validation confusion?
I am using K-Fold cross validation to test my trained model.But i was amazed that for every K-fold the accuracy is different.For instance if use 5-K fold ,every fold has different accuracy.So which ...
2
votes
0answers
69 views
How to randomly sample crops from plain image with points only if crop contains n points inside?
Lets assume I have an image which only has white background and black points, all same size. I need to randomly sample crops with a hardcoded size. The condition is that all the crops need to contain ...
-1
votes
1answer
168 views
How to Choose a Sample for Multiply Classifiers
I've got a dataset of 1.5 million and am looking to train 7 different classifiers -- for each classifier I have up to 10 classes to predict. The total sample has 20K text features (more if I include ...
1
vote
0answers
18 views
Calculate accuracy of crowdsourced responses in realtime
I have a total dataset of, lets say, 1000 Items. Each item will get a response from the crowd.
After I get all the 1000 responses, I can sample the dataset to calculate accuracy. Sampling would ...
