Predicting deterioration of equipment on a production line

Question

Background

There is a production line where there is a machine that interacts with some tools. The process goes as such.

1. Machine makes a product
2. Product is moved into the tool [tool here is essentially a box the condition of which deteriorates over time with use]

When the product is moved into a tool, there is a failure where this transformation fails. Hence there is an unsuccessful transfer of the product into the tools. This is a fault.

This failure occurs because overtime, there is a gradual decline in the quality of the tool. I have built an analytics system where I can see which tool is faulting the most and capture each failure and the associated tool ID.

Problem Statement

Great, now I can identify the damaged tools and fix them. This raises the question of How many tools do I need to fix per month to keep the number of faults at an acceptable level?. But how do we turn this question into a mathematical problem that can be solved?

Currently, on average, each tool has 20 faults per year however around >30% of the tools have 40+ faults per year. One approach could be to set 'limits'. So for example, if a tool has 20 faults per year, this is OK but if it has 40+ faults per year or is on track to achieve this, then it needs to be fixed.

This method isn't the best due to various reasons. Therefore, I would like to build a model which is able to tell me which tools are in bad condition and hence needs to be fixed. I believe a good way to do this is to create a multiclass classifier where per given tool, it can be classified as 'good' , 'OK' or 'bad' and the bad needs to be fixed. Are there better ways of approaching this?

The Data I Have

Initially I have a table which captures the following information. For example, for the instance that tool 104 was in use (this could be an hour, 2 hours, 4 hours etc, depends) it had 1 fault. There could be multiple faults per instance a certain tool was used and similarly this could be zero.

In addition to this, I can have another column which specified the datetime that specific failure occurred.

Tool ID	Faults	Datetime
104	1	26/07/2022 16:00:00
321	2	21/07/2022 04:00:00
1043	0	2/04/2022 12:00:00
321	2	6/01/2022 08:00:00
.......	....	.....

This information can then be 'grouped' where for example, for a given time frame, all instances that Tool ID 104 was used, we will have one row with the Tool ID and all the faults that happened.

For most tools, there is not much data as they are 'good' however for the 30% of tools that have many faults, they may have around 40 data points per year. The best model should be able to classify a bad tool as soon as possible so it can be fixed asap before it causes many faults.

How do we solve this problem?

The data is not exactly normally distributed and infact doesn't really fit any distribution (Here is the histogram for total faults per tool id for a given time frame) -

What is the appropriate model to use for such a problem and the best approach to start working on this?

Answer 1

From how you stated the problem it looks safe to assume that the rate of faults (# faults in instance/instance length) in a tool is a function of how long it has been operating since last fixed, as well as possibly differences in the quality of the tool itself.

A straightforward way to encode those assumptions would be to assume that for every instance that the tool is working, the rate of faults $Y$ is distributed as Poisson with the time the tool was operated so far $t$:

$$\mathbb{E}(Y)=e^{\beta_i + \beta_t}$$

(where $\beta_i$ is the fault rate at time 0 for tool $i$)

The above is also called "Poisson regression" (there's lot's of implementations, for example the glm function in r)

Depending on how much data you have for every tool you could let $\beta_t$ also vary by tool.

I believe a good way to do this is to create a multiclass classifier where per given tool, it can be classified as 'good' , 'OK' or 'bad' and the bad needs to be fixed. Are there better ways of approaching this?

Depending on how you run maintenance it may be a better idea to rank tools according to $\beta_i + \beta_t \cdot t_i$ where $t_i$ is the time that tool $i$ has been operating so far. Then, instead of deciding on an arbitrary threshold for "good" vs "bad" you could let them fix how many tools they can manage and your model would serve to prioritize.

A small simulation study

This should make the above suggestion more concrete.

library(dplyr)
library(ggplot2)
# you can play around with the parameters below to better match your data
# If you have a very high percentage of 0 faults than you'd probably need to 
# use negative binomial regression instead. But this could be a good start.

N <- 100
beta <- data.frame(i = factor(1:N), beta_i = rgamma(n = N, shape=4, scale = 0.05))
summary(beta$beta_i)
beta_t <- 0.002

session_dat <- data.frame(i = factor(unlist(sapply(1:N, function(i) rep(i, sample(100:150, size = 1)))))) %>% 
  mutate(instance_length = sample(2:8, size = nrow(.), replace = T)) %>% 
  group_by(i) %>% 
  mutate(time_operated_till_instance = lag(cumsum(instance_length)), 
         instance_id = 1:n()) %>% 
  ungroup() %>% 
  replace(is.na(.),0) %>% 
  left_join(beta, by = "i") %>% 
  mutate(lambda = exp(log(instance_length) + beta_i + time_operated_till_instance*beta_t), 
         faults = sapply(lambda, function(lam) rpois(1,lam))) %>% 
  select(i, instance_id, instance_length, faults, time_operated_till_instance)

session_dat %>% 
  ggplot(aes(time_operated_till_instance, log(faults))) + 
  geom_point(alpha = 0.05) + stat_smooth()

session_dat %>% filter(i %in% sample(i, 10)) %>% 
  ggplot(aes(time_operated_till_instance, log(faults), color = i)) + 
  geom_point(alpha = 0.1) + stat_smooth(se = F)

poiss_reg <- glm(faults ~ offset(log(instance_length)) + i + time_operated_till_instance - 1, 
                 data = session_dat, family=poisson(link=log))
poiss_reg

# If you look at the coefficient for `time_operated_till_instance`
# you'll see it's pretty close to `beta_t`. The other betas are wobly, 
# but could be used as a somewhat noisy estimate of the tools quality.

Answer 2

It could be regarded as a predictive maintenance problem: The aim is to detect when the tool has to be repaired or replaced before it makes too many faults.

That's why you can detect the intervals between faults: the closer they are, the more the tool is damaged. Thanks to this information, you could alert to repair or change the tool. Algorithms like Random Forest or XGBoost can easily detect when the situation is critical if you have enough data with the whole tool service time (= until there are too many faults).

Like any predictive maintenance project, it is possible to detect the main root causes and see if an action could be taken before the situation worsens. This is possible if you classify faults into categories.

In terms of algorithms, multi-class predictors are a good option, but they work better if there are several correlated features.

Random Forest:

https://www.kaggle.com/code/irajahangari/random-forest-for-predictive-maintenance

https://github.com/Yi-Chen-Lin2019/Predictive-maintenance-with-machine-learning/blob/master/supervised_learning_failure_prediction.ipynb

XGBoost:

https://github.com/aws-samples/amazon-sagemaker-predictive-maintenance-deployed-at-edge/blob/master/predictive-maintenance-xgboost.ipynb

https://github.com/iameminmammadov/dash-predictive-maintenance

https://medium.com/swlh/machine-learning-for-equipment-failure-prediction-and-predictive-maintenance-pm-e72b1ce42da1

Answer 3

infact doesn't really fit any distribution

Have you considered Poisson distribution? The problem sounds like it's general use case. An example of the task using R glm and more

2. I assume you have more data that just dates of each accidents for each tool id. Otherwise, I'm not sure I'll be able to predict a lot. Under the assumption, you may convert the task from regression to classification trying to predict "the tool will have more than 20 faults or not".

3. It can be a computer vision problem. The system that tracks the degree of wear visually if the tool guts are visible. In some situations you may use a microphone and detect a bad system state auditory. I have a lot of friends who may detect problems with a car engine this way so I'm sure this information exists in some mechanical systems.There are some other interesting examples

4. Imagine a case: you have a saw with a small crack. If you have this information in your data you'll be able to predict the saw is worn out and is going to break down soon. This information "there is a crack" is essential for a good prediction, you'll barely be able to extract it's analogue from other saw characteristics like color or length. Maybe you're able to colllect more relevant data

Answer 4

There are many ways to frame your problem.

One way is as a beta regression, a beta regression predicts values between 0 and 1. The target variable would be the probability of failure. The features would be all the properties of the tools (e.g., time since the last failure, total number of failures, product components, who used a tool, …). After fitting a beta regression model, the trained model would predict the probability of failure given the current values of the features. Then the tools could be rank ordered for maintenance.

Another issue is the level of granularity at which to fit the model. You can fit a global model for all tools, a couple of different models based on tool similarity, or an individual model per tool. If you have enough data, one model per tool would be the most precise. If not tools could be grouped together. You mention there are "good" and "problematic" tools. Often there is not enough data for individual or segmented models, so a single model is fit to all the data.

Answer 5

As pointed out above there is many ways to model your problem.

Consider survival regression https://lifelines.readthedocs.io/en/latest/Survival%20Regression.html#model-probability-calibration and https://scikit-survival.readthedocs.io/en/stable/api/generated/sksurv.tree.SurvivalTree.html for predicting the failure probability over time.

You could also try to predict if the machine falls into the group having less than 20 failures per year or if it falls into the other group.