3
$\begingroup$

Background

There is a production line where there is a machine that interacts with some tools. The process goes as such.

  1. Machine makes a product
  2. Product is moved into the tool [tool here is essentially a box the condition of which deteriorates over time with use]

When the product is moved into a tool, there is a failure where this transformation fails. Hence there is an unsuccessful transfer of the product into the tools. This is a fault.

This failure occurs because overtime, there is a gradual decline in the quality of the tool. I have built an analytics system where I can see which tool is faulting the most and capture each failure and the associated tool ID.

Problem Statement

Great, now I can identify the damaged tools and fix them. This raises the question of How many tools do I need to fix per month to keep the number of faults at an acceptable level?. But how do we turn this question into a mathematical problem that can be solved?

Currently, on average, each tool has 20 faults per year however around >30% of the tools have 40+ faults per year. One approach could be to set 'limits'. So for example, if a tool has 20 faults per year, this is OK but if it has 40+ faults per year or is on track to achieve this, then it needs to be fixed.

This method isn't the best due to various reasons. Therefore, I would like to build a model which is able to tell me which tools are in bad condition and hence needs to be fixed. I believe a good way to do this is to create a multiclass classifier where per given tool, it can be classified as 'good' , 'OK' or 'bad' and the bad needs to be fixed. Are there better ways of approaching this?

The Data I Have

Initially I have a table which captures the following information. For example, for the instance that tool 104 was in use (this could be an hour, 2 hours, 4 hours etc, depends) it had 1 fault. There could be multiple faults per instance a certain tool was used and similarly this could be zero.

In addition to this, I can have another column which specified the datetime that specific failure occurred.

Tool ID Faults Datetime
104 1 26/07/2022 16:00:00
321 2 21/07/2022 04:00:00
1043 0 2/04/2022 12:00:00
321 2 6/01/2022 08:00:00
....... .... .....

This information can then be 'grouped' where for example, for a given time frame, all instances that Tool ID 104 was used, we will have one row with the Tool ID and all the faults that happened.

For most tools, there is not much data as they are 'good' however for the 30% of tools that have many faults, they may have around 40 data points per year. The best model should be able to classify a bad tool as soon as possible so it can be fixed asap before it causes many faults.

How do we solve this problem?

The data is not exactly normally distributed and infact doesn't really fit any distribution (Here is the histogram for total faults per tool id for a given time frame) -

enter image description here

What is the appropriate model to use for such a problem and the best approach to start working on this?

$\endgroup$
6
  • $\begingroup$ I'm interested in the question but can't understand and visualize (maybe because I'm not a native English speaker) what means this abstract "Product is moved into the tool". Then you say "this transformation fails". How moving can be transformation? Is it possible to create a real world example in two-three sentences, even a different one. Like: product = nail, tool = hammer? What is transformation if we already has a product? $\endgroup$ Jul 26, 2022 at 12:44
  • 1
    $\begingroup$ @SergeySkripko Hi Sergey, the 'tools' mentioned here are essentially large boxes. Think of it as a machine which makes a product. This is then moved from the machine into this box using various instruments. It is like going into a grocery store, finding something you like on a shelf, picking it up with your hand, and then putting it in your shopping trolley. Where the shelf with that item is the machine and the shopping trolley is your Tool. $\endgroup$
    – fred
    Jul 26, 2022 at 13:27
  • 1
    $\begingroup$ The transformation is a movement transformation. the product's position is changed while moving it into the box $\endgroup$
    – fred
    Jul 26, 2022 at 13:39
  • $\begingroup$ one idea is to use similated data by randomly sampling your dataset and attempt to create a normal distribution then your predictions will not be random. $\endgroup$ Jul 26, 2022 at 15:16
  • $\begingroup$ I don't quite understand the data: Is a tool recorded every time if it was in use even if there was no fault? The statetment "For most tools, there is not much data as they are 'good'" would make me think that this is not the case, however the table seems to indicate that it is. $\endgroup$ Sep 1, 2022 at 9:53

5 Answers 5

3
+100
$\begingroup$

From how you stated the problem it looks safe to assume that the rate of faults (# faults in instance/instance length) in a tool is a function of how long it has been operating since last fixed, as well as possibly differences in the quality of the tool itself.

A straightforward way to encode those assumptions would be to assume that for every instance that the tool is working, the rate of faults $Y$ is distributed as Poisson with the time the tool was operated so far $t$:

$$\mathbb{E}(Y)=e^{\beta_i + \beta_t}$$

(where $\beta_i$ is the fault rate at time 0 for tool $i$)

The above is also called "Poisson regression" (there's lot's of implementations, for example the glm function in r)

Depending on how much data you have for every tool you could let $\beta_t$ also vary by tool.

I believe a good way to do this is to create a multiclass classifier where per given tool, it can be classified as 'good' , 'OK' or 'bad' and the bad needs to be fixed. Are there better ways of approaching this?

Depending on how you run maintenance it may be a better idea to rank tools according to $\beta_i + \beta_t \cdot t_i$ where $t_i$ is the time that tool $i$ has been operating so far. Then, instead of deciding on an arbitrary threshold for "good" vs "bad" you could let them fix how many tools they can manage and your model would serve to prioritize.

A small simulation study

This should make the above suggestion more concrete.

library(dplyr)
library(ggplot2)
# you can play around with the parameters below to better match your data
# If you have a very high percentage of 0 faults than you'd probably need to 
# use negative binomial regression instead. But this could be a good start.

N <- 100
beta <- data.frame(i = factor(1:N), beta_i = rgamma(n = N, shape=4, scale = 0.05))
summary(beta$beta_i)
beta_t <- 0.002

session_dat <- data.frame(i = factor(unlist(sapply(1:N, function(i) rep(i, sample(100:150, size = 1)))))) %>% 
  mutate(instance_length = sample(2:8, size = nrow(.), replace = T)) %>% 
  group_by(i) %>% 
  mutate(time_operated_till_instance = lag(cumsum(instance_length)), 
         instance_id = 1:n()) %>% 
  ungroup() %>% 
  replace(is.na(.),0) %>% 
  left_join(beta, by = "i") %>% 
  mutate(lambda = exp(log(instance_length) + beta_i + time_operated_till_instance*beta_t), 
         faults = sapply(lambda, function(lam) rpois(1,lam))) %>% 
  select(i, instance_id, instance_length, faults, time_operated_till_instance)

session_dat %>% 
  ggplot(aes(time_operated_till_instance, log(faults))) + 
  geom_point(alpha = 0.05) + stat_smooth()

enter image description here

session_dat %>% filter(i %in% sample(i, 10)) %>% 
  ggplot(aes(time_operated_till_instance, log(faults), color = i)) + 
  geom_point(alpha = 0.1) + stat_smooth(se = F)

enter image description here

poiss_reg <- glm(faults ~ offset(log(instance_length)) + i + time_operated_till_instance - 1, 
                 data = session_dat, family=poisson(link=log))
poiss_reg

# If you look at the coefficient for `time_operated_till_instance`
# you'll see it's pretty close to `beta_t`. The other betas are wobly, 
# but could be used as a somewhat noisy estimate of the tools quality.
$\endgroup$
7
  • $\begingroup$ Thank you for your input. This is a great way of approaching it but there are some significant issues with this. The deterioration of a tool is not necessarily time dependent. I could have a tool that was fixed yesterday be completely damaged today if it jammed or was mishandled by operators etc, there are many other factors involved. Additionally I have a 1000 tools. This means I would need to create a poisson distribution for each individual one to accurately be able to determine the behavior of each tool as their physical make up over time has changed and therefore have $\endgroup$
    – fred
    Sep 1, 2022 at 12:11
  • $\begingroup$ different characteristics. There is not enough data (on avg 20 data points per year per tool) for each individual tool to make a distribution for them individually. $\endgroup$
    – fred
    Sep 1, 2022 at 12:11
  • $\begingroup$ @fred "I could have a tool that was fixed yesterday be completely damaged today if it jammed or was mishandled by operators etc" - Well you agree that the probability of such mishaps increases with time right? If so, then the number of faults is an increasing function of time. $\endgroup$
    – Iyar Lin
    Sep 1, 2022 at 12:42
  • 1
    $\begingroup$ You encode the difference between tools with beta_i, you don't need a seperate model for each tool. I'll a small simulation analysis to illustrate $\endgroup$
    – Iyar Lin
    Sep 1, 2022 at 12:58
  • 1
    $\begingroup$ Because the NB model is more flexible. You could also deal directly with zero inflated data using zaro inflated poisson/BN models. See this nice post for more: towardsdatascience.com/… $\endgroup$
    – Iyar Lin
    Sep 4, 2022 at 11:46
2
$\begingroup$

It could be regarded as a predictive maintenance problem: The aim is to detect when the tool has to be repaired or replaced before it makes too many faults.

That's why you can detect the intervals between faults: the closer they are, the more the tool is damaged. Thanks to this information, you could alert to repair or change the tool. Algorithms like Random Forest or XGBoost can easily detect when the situation is critical if you have enough data with the whole tool service time (= until there are too many faults).

Like any predictive maintenance project, it is possible to detect the main root causes and see if an action could be taken before the situation worsens. This is possible if you classify faults into categories.

In terms of algorithms, multi-class predictors are a good option, but they work better if there are several correlated features.

Random Forest:

https://www.kaggle.com/code/irajahangari/random-forest-for-predictive-maintenance

https://github.com/Yi-Chen-Lin2019/Predictive-maintenance-with-machine-learning/blob/master/supervised_learning_failure_prediction.ipynb

XGBoost:

https://github.com/aws-samples/amazon-sagemaker-predictive-maintenance-deployed-at-edge/blob/master/predictive-maintenance-xgboost.ipynb

https://github.com/iameminmammadov/dash-predictive-maintenance

https://medium.com/swlh/machine-learning-for-equipment-failure-prediction-and-predictive-maintenance-pm-e72b1ce42da1

$\endgroup$
0
$\begingroup$

infact doesn't really fit any distribution

Have you considered Poisson distribution? The problem sounds like it's general use case. An example of the task using R glm and more

  1. I assume you have more data that just dates of each accidents for each tool id. Otherwise, I'm not sure I'll be able to predict a lot. Under the assumption, you may convert the task from regression to classification trying to predict "the tool will have more than 20 faults or not".

  2. It can be a computer vision problem. The system that tracks the degree of wear visually if the tool guts are visible. In some situations you may use a microphone and detect a bad system state auditory. I have a lot of friends who may detect problems with a car engine this way so I'm sure this information exists in some mechanical systems.There are some other interesting examples

  3. Imagine a case: you have a saw with a small crack. If you have this information in your data you'll be able to predict the saw is worn out and is going to break down soon. This information "there is a crack" is essential for a good prediction, you'll barely be able to extract it's analogue from other saw characteristics like color or length. Maybe you're able to colllect more relevant data

$\endgroup$
5
  • $\begingroup$ 1) The reason I do not want to use a Poisson distribution is because it assumes the events are independent and I can tell you that there are other factors involved which has a slight effect on these faults and hence are not entirely independent. 2) I also have data on exact time stamps within that day as aswell as how long each tool spent in the machine. I have about 100 other variables of data for this process. $\endgroup$
    – fred
    Jul 28, 2022 at 7:42
  • $\begingroup$ 3) What exactly do you mean by tool guts? hmm this is a new way of looking at the problem. very interesting. I think it would be near impossible to use the sounds->image ideas since the location of these machine is very very loud and a mic will pick up a lot of noise. $\endgroup$
    – fred
    Jul 28, 2022 at 7:43
  • $\begingroup$ 4) I completely agree with your point here. To get to a point where I can finance the instruments to capture more relevant data that specifically captures this information, I would need to know exactly why this happens and I dont have the data to prove that right now. However, having said that, I still have a LOT of data where I dont even know where to start with $\endgroup$
    – fred
    Jul 28, 2022 at 7:46
  • $\begingroup$ ok, no sound. Some wifi accelerometer? It may depend on the tool but if it vibrates in different modes depending on the worn out. I'm really far from physics. Or a thermometer if the deterioration causes more/less friction. You get the direction of thought $\endgroup$ Jul 28, 2022 at 13:36
  • $\begingroup$ or maybe classification as I described above. Classification "does it break down today/tomorrow"? Or regression "how many days left with the current tool state?" $\endgroup$ Jul 28, 2022 at 13:38
0
$\begingroup$

There are many ways to frame your problem.

One way is as a beta regression, a beta regression predicts values between 0 and 1. The target variable would be the probability of failure. The features would be all the properties of the tools (e.g., time since the last failure, total number of failures, product components, who used a tool, …). After fitting a beta regression model, the trained model would predict the probability of failure given the current values of the features. Then the tools could be rank ordered for maintenance.

Another issue is the level of granularity at which to fit the model. You can fit a global model for all tools, a couple of different models based on tool similarity, or an individual model per tool. If you have enough data, one model per tool would be the most precise. If not tools could be grouped together. You mention there are "good" and "problematic" tools. Often there is not enough data for individual or segmented models, so a single model is fit to all the data.

$\endgroup$
0
$\begingroup$

As pointed out above there is many ways to model your problem.

Consider survival regression https://lifelines.readthedocs.io/en/latest/Survival%20Regression.html#model-probability-calibration and https://scikit-survival.readthedocs.io/en/stable/api/generated/sksurv.tree.SurvivalTree.html for predicting the failure probability over time.

You could also try to predict if the machine falls into the group having less than 20 failures per year or if it falls into the other group.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.