Skip to main content

Questions tagged [data-leakage]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

Understanding the data leakage article on The ICML 2013 Whale Challenge - Right Whale Redux

I came across this article while trying to understand more about data leakage. https://www.kaggle.com/competitions/the-icml-2013-whale-challenge-right-whale-redux/discussion/4865 Though really good ...
sarah's user avatar
  • 1
0 votes
0 answers
13 views

Objective function in Bayesian Hyperparameter Tuning

I have a question that has been going around in my head for a while and I'd like to leverage the wisdom of the crowd for getting a few opinions on it. Let me describe the Problem: I have a relatively ...
Hive5's user avatar
  • 1
6 votes
1 answer
674 views

How high of a correlation coefficient of a feature with a target variable is considered too high?

Currently my classification model is doing too well on all of the train, validation, and test datasets. I'm assuming there is a data leakage in the features, and therefore I've computed the ...
haneulkim's user avatar
  • 469
1 vote
1 answer
278 views

What is the best way to avoid data leaking in timeseries forecasting multiple labels?

I'm predicting crypto pairs volume. I want to increase my accuracy by using one model for different pairs. Question is how to avoid timedata leaking? Example n - pairs, t- time, m - features for each. ...
Dima's user avatar
  • 11
0 votes
0 answers
27 views

Computer Vision - is splitting by annotation considered data leakage?

We are using a COCO like dataset, with around 1500 images and 4700 annotations. We want to define our train-test dataset split. Will splitting our dataset by annotations cause data leakage? We fear ...
Toffe1369's user avatar
0 votes
0 answers
167 views

Why does Time series split cause data leakage from future data?

There are lots of websites saying time series split may cause data leakage. The idea for time series splits is to divide the training set into two folds at each iteration on condition that the ...
Ellen's user avatar
  • 1
0 votes
1 answer
50 views

Is this a case of leakage or not?

I have a set of data on individuals' performance in 1960,1970,1980 and 1990, e.g. chess rating in those years for a bunch of players with 40-year careers. I've been asked to build a model to predict ...
jeremy_rutman's user avatar
0 votes
1 answer
52 views

Is it unethical to gather data from data leaks about demographics?

Sorry if this is the wrong SE, but in my mind it made the most sense to ask this here. My question is related to specifically collecting information about a target demographic, not individuals which ...
Justin T's user avatar
  • 101
1 vote
2 answers
517 views

what qualifies as a data leakage?

I am currently working on a binary classification problem using imbalanced data. The algorithm that I am using is random forest. The problem is about predicting whether each sales project will meet ...
The Great's user avatar
  • 2,575
1 vote
1 answer
49 views

Order of preproccesing, avoiding leakage and metrics

I have a dataset with ~40k records and 16 columns (including the target) and I want to understand the correct process behind whole data science proccess. This is what I did: Performed an EDA which ...
pustelnikk's user avatar
0 votes
1 answer
147 views

Fit multiple models e.g classifiers -> stacking -> calibration without data-leak or getting too many datasets

I have some data X on which I want to do the following: Train two models; SVM and Logistic Regression Use a stacking classifier based on the models from (1) ...
CutePoison's user avatar
0 votes
1 answer
56 views

using a feature that is only available during training

I'm working on a project that aims to classify JIRA issues into their relevant owner group. An issue has the following text features: Summary Description Comments all of which are text based. During ...
Ben's user avatar
  • 209
0 votes
1 answer
28 views

Data Leakage possibility related to replacing rare values on a categorical column

I am exploring the possibility of data leakage for categorical columns, replacing rare values with category 'Other' Let's say I have a DF with 40 categorical columns. I will check each of them, find ...
Ali Kılınç's user avatar
1 vote
1 answer
194 views

Revisit - Encoding before vs after train test split?

Background: Like a good data scientist, I planned to fit my encoder on my training set and use it to transform my test set. However, when I tried to transform my test set, the encoder threw an error....
Snehal Patel's user avatar
1 vote
1 answer
54 views

Can using model evaluation metrics to choose a model cause data leakage?

I was reading a blog post about improving machine-learning model train/validate/test splits. Towards the end was this remark: I say we should be more creative in the way we test machine learning ...
Davis's user avatar
  • 119
1 vote
0 answers
66 views

Data Leakage (specific case with standardization)

I read a few blog posts and found some books that caution against data leakage when adjusting the training data using any information from the held out datasets. For example, if I were to standardize ...
dzheng1887's user avatar
2 votes
2 answers
387 views

Stacking: Use predictions of train or test to create features for level 1 classifier

The question is pretty simple. In stacking, the predictions of level 0 models are being used as features to train a level 1 model. However, the predictions of what data? Intuitively it makes more ...
liakoyras's user avatar
  • 636
1 vote
1 answer
54 views

Machine learning data leakage

I have a dataset with Information about a transaction that a customer made in the store. the target is Indication of whether fraud was committed in the transaction. One of my feature is ...
Yochai 's user avatar
1 vote
1 answer
111 views

Avoid leakage in NLP extraction

What is best practice for applying traditional NLP extraction techniques a pre-processing for ML models? Given a pipeline: Collect raw data. Parse full data set with a variety of traditional NLP ...
GooJ's user avatar
  • 435
1 vote
1 answer
246 views

Why leaky features are problematic

I want to know why leaky features are problematic in machine learning/data science. I'm reading a book that uses the Titanic dataset for illustration. It says that the column ...
Huy Truong's user avatar
2 votes
1 answer
56 views

Can I create a new target value based on the average target value of same data points for regression?

I am trying to predict profit of retail stores. The orginal dataframe looks like this: Store No feature A feature B year profit A 1 2 2016 20000 A 1 2 2017 40000 B 4 3 2017 50000 B 4 3 2018 40000 ...
freshst4r's user avatar
2 votes
0 answers
128 views

Is data leakage giving me misleading results? Independent test set says no!

TLDR: I evaluated a classification model using 10-fold CV with data leakage in the training and test folds. The results were great. I then solved the data leakage and the results were garbage. I then ...
PeMADS's user avatar
  • 21
2 votes
1 answer
138 views

How does a data leakage work?

I'm working with panel data - every row represents a timestamp (observation) and there are multiple rows for a single timestamp (around 20 rows each). I have a total of 8719 unique timestamps. ...
UgurZCifci's user avatar
1 vote
2 answers
83 views

How to generate test set with no data-leakage using multiple columns

I am developing a fraud detection algorithm. Among other things, my dataset contains the phone number, email address and a few other fields that should uniquely identify a user (let's call them "...
Anatole's user avatar
  • 181
0 votes
2 answers
1k views

Why label encoding before split is data leakage?

I want to ask why Label Encoding before train test split is considered data leakage? From my point of view, it is not. Because, for example, you encode "good" to 2, "neutral" to 1 ...
Anar's user avatar
  • 73
-1 votes
2 answers
2k views

How to fit Word2Vec on test data?

I am working on a Sentiment Analysis problem. I am using Gensim's Word2Vec to vectorize my data in the following way: ...
spectre's user avatar
  • 2,115
1 vote
1 answer
63 views

Preprocessing for the final model to be deployed

Typically for a ML workflow, we import the data (X and y), split the X and ...
spectre's user avatar
  • 2,115
4 votes
1 answer
222 views

What can I do when my test and validation scores are good, but the submission is terrible?

This is a very broad question, I understand and I'm totally fine if someone believes it's not appropriate to do it. But it's killing me not to understand this... Here's the thing, I'm doing a machine ...
Yuxxxxxx's user avatar
  • 141
0 votes
1 answer
2k views

Splitting before tfidf or after?

When should I perform preprocessing and matrix creation of text data in NLP, before or after train_test_split? Below is my sample code where I have done ...
spectre's user avatar
  • 2,115
3 votes
1 answer
147 views

Can data leakage be sometimes acceptable?

I have recently started using kaggle and I have stumbled on a few examples of practices I would consider do be data leakage. Many of them were done by people well established on the platform and I ...
Mateusz's user avatar
  • 125
1 vote
1 answer
274 views

Data Leakage when preprocessing categorical features?

I am fairly new to machine learning. I came across the concept of Data Leakage. The article says that always split the data before performing preprocessing steps. My question is, do steps such as ...
Joe's user avatar
  • 85
1 vote
1 answer
386 views

Is there potentially data leakage during imputation for time-varying sensor data?

I have a time-varying dataset that contains some missing data. I have sensors that continuously monitor some properties at evenly-spaced intervals and I would like to impute the missing values using ...
Evan's user avatar
  • 185
2 votes
1 answer
121 views

Information leakage when train/test are truly i.i.d.?

I am well aware that to avoid information leakage, it is recommended to fit any transformation (e.g., standardization or imputation based on the median value) on the training dataset and applying it ...
thereandhere1's user avatar
1 vote
2 answers
223 views

Does binning a time series with pd.qcut (using quantiles) create data leakage?

Let's say I want to predict whether a company will default on it's debt at some point in time (so binary classification) and one of the time series variables I'm using is the "revenue" of ...
Alexandre Martin's user avatar
1 vote
0 answers
74 views

Data\Feature Leakage - feature too close to target?

In General: The target itself built from very correlated features, because there is no ground truth - only rule based one. I have a problem in the following method: Output: binary. built from ...
orbgr's user avatar
  • 21
1 vote
1 answer
1k views

K-Fold cross validation and data leakage

I want to do K-Fold cross validation and also I want to do normalization or feature scaling for each fold. So let's say we have k folds. At each step we take one fold as validation set and the ...
user avatar
2 votes
1 answer
152 views

How to split up my dataset in a train and testset, in order to prevent data leakage?

I realize that this could be considered a duplicate of this question, Is using samples from the same person in both trainset and testset considers being a data leakage?, where it is stated that "...
JonasL's user avatar
  • 21
1 vote
1 answer
347 views

Is it right to maintain the train distribution in test set for unbalanced data?

If the training set was unbalanced the chances are the model will be biased. But if the data distribution in the test set is the same distribution as the train set, this kind of bias is not going to ...
Marzi Heidari's user avatar
7 votes
4 answers
3k views

Does label encoding an entire dataset cause data leakage?

I have a dataset on which one of the features has a lot of different categorical values. Trying to use a LabelEncoder, OrdinalEncoder or a OneHotEncoder results in an error, since when splitting the ...
kaylani2's user avatar
2 votes
1 answer
70 views

Is using samples from the same person in both trainset and testset considers being a data leakage?

Suppose a neural network is built for a binary classification problem such as recognize the face as a smiley face or not, by using a dataset of 1000 persons and each person has ten images of his face. ...
AI_new2's user avatar
  • 85
2 votes
0 answers
332 views

Will setting up time series data in this way cause data leakage?

I am trying to predict future stock market values using a gradient boosted tree model. As far as I know, gradient boosted trees use the data in one row, and only that row, to predict the target ...
Darcey BM's user avatar
  • 197
2 votes
1 answer
126 views

Can I apply feature selection before splitting by requiring selection occurs > 90% of time

I want to move the feature selection step to before splitting to save time and allow bigger input dataset. If, in repeated subsamples, a feature is selected in over X percentage of cases I will keep ...
ran8's user avatar
  • 345
1 vote
1 answer
383 views

Mean encoding in times series

Considering a TS from 1-5 blocks and using 1-4 block as train data. Is it invalid to build mean encode on whole train data / or I should mean encode block 1 / block 1-2 / ... / block 1-2-3-4 ? Edit 1 :...
alexandre_d's user avatar
1 vote
2 answers
544 views

Need help understanding data leakage

I am a newbie to this stuff so I am sorry if my question is stupid~ I need help understanding what data leakage between X_train and X_test is and when exactly it happens. I am currently working on a ...
moii789's user avatar
  • 73
1 vote
1 answer
35 views

Normalizing dependent feature by one of the independent ones

I have a data set with three different features (x1, x2, x3) and I am going to use a regression model to predict y based on the features. x3 is the total amount of money that a customer invest and y ...
nimar's user avatar
  • 758
1 vote
0 answers
94 views

Identifying possible data leakage

I am building a binary classification model for imbalanced dataset using XGBoost. I tuned the hyperparameters for four different models based on 2 training datasets and 2 optimization metrics. Class ...
thereandhere1's user avatar
3 votes
0 answers
455 views

Data leakage in bidirectional LSTM timeseries data

Does it cause data leakage to train a bidirectional LSTM on data where a user can be a sample in the training data multiple times? Each row is a snapshot at a different point in time for a given ...
David Feldman's user avatar
1 vote
0 answers
37 views

Is data leakage in time series due to both I's of the IID principe or only one?

I am sure that the Independent part of the IID principle gives you data leakage because of the correlation. But the identical part I am not so sure. Identical in time series means that your data is ...
Tibo Geysen's user avatar
4 votes
2 answers
37 views

Does using user-specific accumulative variables causes data leakage?

Let's say I have a scenario in which my observational unit is a bill that was issued after a certain service was given and my goal is to predict if this bill is going to be paid or not. I have users ...
Corel's user avatar
  • 159
1 vote
1 answer
434 views

How to keep the test data from leaking into the training process of a machine learning algorithm?

I read in many different sources that I need to split my data into a training set and a test set. Then I have to make sure that the algorithm is trained only on the training data, and do my best to ...
Vladislav Gladkikh's user avatar