Questions tagged [data-drift]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers

Decision tree for selecting tests to monitor ML drift?

I'm exploring concepts for monitoring drift and I have come up with the following decision tree: Criteria Decision Nature of Data Distribution KS test is suitable for comparing continuous ...
Chris Snow's user avatar
0 votes
0 answers

Detecting types of Drifts in data

Given I have two data files, is it possible to detect between the covariate, concept and label drift? That has occurred between them? There are various methods to detect concept drift. But is it ...
pyComali's user avatar
0 votes
0 answers

Datasets for demonstrating data/model/concept drift

Are there any well-known datasets that are useful for illustrating the concept of data and/or model/concept drift? I can create my own out of, say, MNIST (by e.g. removing digits and adding them in ...
Jennie Lan Chien's user avatar
1 vote
1 answer

Do you have to use clustering with SciKit-Learn's Mutual Information metric?

I'd like to calculate the mutual information between two datasets, but I'd prefer not to cluster them first. I'm thinking of using SciKit-Learn's mutual_info_score ...
Connor's user avatar
  • 631
1 vote
0 answers

Very different behaviors between PSI and KS tests for data drift

I want to set up a process for data drift and I am trying to see which metric to pick. Using KS test, it flags half of my features to be drifted although when I look at the distribution between the ...
Fatima's user avatar
  • 71
4 votes
3 answers

What techniques are used to analyze data drift?

I've created a model that has recently started suffering from drift. I believe the drift is due to changes in the dataset but I don't know how to show that quantitatively. What techniques are ...
Connor's user avatar
  • 631
1 vote
1 answer

How train a pre-trained model based on new dataset?

I have trained a deep nn model based on some existing data. In the meantime, I have collected more data and label them so that I can feed it to the model to improve its performance. The questions is, ...
Mahdi Amrollahi's user avatar
1 vote
1 answer

What are the advantages of model drift vs concept drift in online learning?

I have asked this question here but I'm also posting it here to get a better insight:
Ash's user avatar
  • 129
3 votes
1 answer

What is the difference between Covariate Shift, Label Shift, Concept Shift, Concept Drift, and Prior Probability Shift?

As a beginner in MLOps, I was overwhelmed by some confusing definitions. As far as I understand, when we have a classifier or regressor with y = f(X) function: <...
Mohsen Mahmoodzadeh's user avatar
0 votes
1 answer

Can I use Population Stability Index (PSI) when observations have multiple variables?

I understand from resources like this one that the Population Stability Index (PSI) can be used to test for data drift when a machine learning model is in production. However, the resources I have ...
Fijoy Vadakkumpadan's user avatar
8 votes
1 answer

How to Combat Data Drift

I have customer demographic data that include columns like: age, the first half of the postcode, occupation (there is a defined list of possible occupations), and more. Each month I get a new batch of ...
scott lucas's user avatar
2 votes
0 answers

Detecting Data Drift in Audio Data

For a give set of audio files collected from an industrial process via a microphone, I have extracted suitable features and fed them into a neural network for training a binary classifier as depicted ...
TwinPenguins's user avatar
  • 4,219