Questions tagged [mlops]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

how to define search space over hyperparameters automatically?

I'm trying to automate retrain steps of our ML models. My aim is hyper-parameter tuning with current data (newer performance window) using same algorithm and features on production environment. ...
  • 1
0 votes
0 answers
7 views

References to online service/platform dedicated to "time series prediction"

Do you have any references to online service/platform dedicated to "time series prediction" ? For one product under development, we need to perform frequent (i.e. one per 15 minutes) ...
0 votes
2 answers
55 views

Which skill would be more useful for a data scientist? [closed]

DevOps Full Stack Development.
1 vote
1 answer
33 views

Training a CNN in production on new data

How should I approach training a convolutional neural network in production on new data when I detect model performance degradation due to data or concept drift? Resources like this one and this one ...
0 votes
0 answers
5 views

Metrics for monitoring LDA Model

We use LDA for topic-modelling in production. I was wondering if there are any metrics which we could use to monitor the quality of this model to understand when model starts to perform poorly and we ...
  • 1
0 votes
0 answers
110 views

What is the difference between Covariate Shift, Label Shift, Concept Shift, Concept Drift, and Prior Probability Shift?

As a beginner in MLOps, I was overwhelmed by some confusing definitions. As far as I understand, when we have a classifier or regressor with y = f(X) function: <...
0 votes
0 answers
28 views

Can I use Population Stability Index (PSI) when observations have multiple variables?

I understand from resources like this one that the Population Stability Index (PSI) can be used to test for data drift when a machine learning model is in production. However, the resources I have ...
1 vote
1 answer
53 views

Sustain learning separately - continuous learning

This question is to seek suggestions on how to architect the continuous learning approach in distributed manner. Let me explain the situation: In my classification problem, I have classes which can ...
1 vote
0 answers
5 views

How is model scheduling set up in practice?

I have been working on various machine learning models so far, but never yet on the deployment phase of an ML project. I have vaguely used Apache Airflow and I'm aware that it is a tool for scheduling ...
  • 289
8 votes
1 answer
477 views

How to Combat Data Drift

I have customer demographic data that include columns like: age, the first half of the postcode, occupation (there is a defined list of possible occupations), and more. Each month I get a new batch of ...
0 votes
0 answers
86 views

AWS Stepfunctions using SageMaker ScriptProcessor

Hello Data Science/Engineer folks, I need a help in orchestrating a ML model using AWS Step Functions. There are some tutorials on using StepFunction with Python SDKs, but all of them are focused ...
1 vote
2 answers
49 views

Automate Clustering predictions and RFM metrics

We did a POC for customer segmentation and followed the below approach a) extract data from source system (SAP business objects) b) Use python jupyter notebook to manipulate, merge and group data (...
  • 2,449
0 votes
0 answers
20 views

Productionizing ML model and result distribution

We completed POCs for our project involving classification algorithm (Random Forests) Background This project is about predicting project success rate (predict_proba). Data will be refreshed every 45 ...
  • 2,449
0 votes
0 answers
7 views

Storing meta information from preprocessing step

Let's say I have a preprocessing step which removes some columns, replaces rare categories with the keyword "Other" and so on. All of that is done based on an initial data analysis. Now, I ...
  • 103
0 votes
0 answers
7 views

Local Deployment/Installation of Kubeflow on Windows

I am facing a major problem in installing Kubeflow locally on my Windows 10 Machine. Machine Specs - OS: Windows 10, RAM: 16GB Approaches Tried To Install Microk8s - Not Successful I get Cannot ...
9 votes
2 answers
1k views

MLOps for beginner

I am 1 year old in ML and have been using jupyter notebook to build static models all these days, do some analysis and present my results to the bosses as it was all POC. Now, we would like to scale ...
  • 2,449
4 votes
2 answers
95 views

Meaningfully compare target vs observed TPR & FPR

Suppose I have a binary classifier $f$ which acts on an input $x$. Given a threshold $t$, the predicted binary output is defined as: $$ \widehat{y} = \begin{cases} 1, & f(x) \geq t \\ 0, &...
0 votes
1 answer
22 views

How is model evaluation and re-training done after deployment without ground truth labels?

Suppose I deployed a model by manual labeling the ground truth labels with my training data, as the use case is such that there's no way to get the ground truth labels without humans. Once the model ...
  • 133
0 votes
1 answer
126 views

What is MLOPS - Why & How to get started?

MLOPs has been gaining traction and a lot of fortune 500 companies are creating team specifically for MLOPs.Can anyone help me understand? Why MLOPs is gaining so much traction? How it is different ...
  • 1,789
2 votes
1 answer
101 views

Data preprocessing framework/library alternatives

I am currently working on some python machine learning projects that are soon to be deployed to production. As such, in our team we are interested in doing this the most "correct" way, ...
0 votes
0 answers
27 views

How do I verify and test a machine learning model against reality during time?

As a software engineers we familiar with a concept of testing (unit, integration, e2e) Tests give us a level of confidence about the code and changes in our code. Looks like for ML the "code"...
0 votes
0 answers
59 views

Is my idea of a Feature Store wrong?

Cross-posted on Reddit ML. Should a Feature Store be part of an enterprise data catalog? To me, a feature store seems to be a highly niche data catalog but missing a lot of the benefits of having an ...
0 votes
1 answer
200 views

What features used by CNN model should a feature store actually store? [closed]

According to MLOPs principle, it is recommended to have a feature store. The question is in the context of doing image classification using deep learning models like convolutional neural networks ...
8 votes
1 answer
435 views

MLflow real world experience

Can someone provide a summary of the real world deployment experience of MLflow? We have a few ML models (e.g., LightGBM, tensorflow v2, etc.) and want to avoid framework like SageMaker (due to ...