Questions tagged [data-drift]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
57 views

Decision tree for selecting tests to monitor ML drift?

I'm exploring concepts for monitoring drift and I have come up with the following decision tree: Criteria Decision Nature of Data Distribution KS test is suitable for comparing continuous ...
Chris Snow's user avatar
0 votes
0 answers
25 views

Detecting types of Drifts in data

Given I have two data files, is it possible to detect between the covariate, concept and label drift? That has occurred between them? There are various methods to detect concept drift. But is it ...
pyComali's user avatar
0 votes
0 answers
17 views

Datasets for demonstrating data/model/concept drift

Are there any well-known datasets that are useful for illustrating the concept of data and/or model/concept drift? I can create my own out of, say, MNIST (by e.g. removing digits and adding them in ...
Jennie Lan Chien's user avatar
1 vote
1 answer
80 views

Do you have to use clustering with SciKit-Learn's Mutual Information metric?

I'd like to calculate the mutual information between two datasets, but I'd prefer not to cluster them first. I'm thinking of using SciKit-Learn's mutual_info_score ...
Connor's user avatar
  • 631
1 vote
0 answers
62 views

Very different behaviors between PSI and KS tests for data drift

I want to set up a process for data drift and I am trying to see which metric to pick. Using KS test, it flags half of my features to be drifted although when I look at the distribution between the ...
Fatima's user avatar
  • 71
4 votes
3 answers
412 views

What techniques are used to analyze data drift?

I've created a model that has recently started suffering from drift. I believe the drift is due to changes in the dataset but I don't know how to show that quantitatively. What techniques are ...
Connor's user avatar
  • 631
1 vote
1 answer
38 views

How train a pre-trained model based on new dataset?

I have trained a deep nn model based on some existing data. In the meantime, I have collected more data and label them so that I can feed it to the model to improve its performance. The questions is, ...
Mahdi Amrollahi's user avatar
1 vote
1 answer
145 views

What are the advantages of model drift vs concept drift in online learning?

I have asked this question here but I'm also posting it here to get a better insight: https://stats.stackexchange.com/questions/602282/what-are-the-advantages-of-model-drift-vs-concept-drift-in-online-...
Ash's user avatar
  • 129
3 votes
1 answer
1k views

What is the difference between Covariate Shift, Label Shift, Concept Shift, Concept Drift, and Prior Probability Shift?

As a beginner in MLOps, I was overwhelmed by some confusing definitions. As far as I understand, when we have a classifier or regressor with y = f(X) function: <...
Mohsen Mahmoodzadeh's user avatar
0 votes
1 answer
119 views

Can I use Population Stability Index (PSI) when observations have multiple variables?

I understand from resources like this one that the Population Stability Index (PSI) can be used to test for data drift when a machine learning model is in production. However, the resources I have ...
Fijoy Vadakkumpadan's user avatar
8 votes
1 answer
570 views

How to Combat Data Drift

I have customer demographic data that include columns like: age, the first half of the postcode, occupation (there is a defined list of possible occupations), and more. Each month I get a new batch of ...
scott lucas's user avatar
2 votes
0 answers
199 views

Detecting Data Drift in Audio Data

For a give set of audio files collected from an industrial process via a microphone, I have extracted suitable features and fed them into a neural network for training a binary classifier as depicted ...
TwinPenguins's user avatar
  • 4,219