0
$\begingroup$

I'm studying the behavior of machine failures in a production scenario. For this, I generated random data to form my imbalanced training set, consisting of categorical data, which indicate whether or not there was a failure in each subperiod. The failures were generated according to a exponential distribution. I have 100 features (Period_1 to Period_100), each containing information for 112 subperiods. My intention is to predict the behavior of the failures for the next period. However, I have two questions:

  1. Is a learning algorithm really necessary to predict the next period failures or could some simpler statistical method be used?
  2. The 100 features represent the same type of information, but for different periods. Would I have a problem with that?

Any help will be appreciated.

A sample of the data:

    Period_1 Period_2 Period_3 Period_4 Period_5 Period_6 Period_7 Period_8
1     Normal   Normal   Normal   Normal   Normal   Normal   Normal   Normal
2     Normal   Normal   Normal   Normal   Normal   Normal   Normal   Normal
3     Normal   Normal  Failure   Normal   Normal   Normal   Normal   Normal
4     Normal  Failure   Normal   Normal   Normal   Normal   Normal  Failure
5     Normal   Normal   Normal   Normal   Normal   Normal   Normal   Normal
6     Normal   Normal   Normal   Normal   Normal   Normal   Normal   Normal
7     Normal   Normal   Normal   Normal   Normal   Normal   Normal   Normal
8     Normal   Normal   Normal   Normal   Normal   Normal   Normal   Normal
9     Normal   Normal   Normal   Normal   Normal   Normal   Normal   Normal
10    Normal  Failure   Normal   Normal   Normal   Normal   Normal   Normal

In this moment, I'm using several classification methods to predict the failures: gradient boosting algorithm, random forest, Classification and Regression Trees, neural networks, Bagged CART, SVM, C5.0, eXtreme Gradient Boosting, and k-Nearest Neighbors. For all this, I use strategy to deal with the imbalance.

$\endgroup$
  • $\begingroup$ many machine learning algorithms are statistical in nature, there is no contradiction between the two $\endgroup$ – Nikos M. Jul 28 at 17:41
  • $\begingroup$ Have you been able to get a better score than the one you get by always guessing the more popular behavior (in this case, guessing Normal for everything in the next period)? I don't see what function a machine learning algorithm could learn, since you generated the data using an exponential distribution. Is there any reason not to just model the process as an exponential distribution? $\endgroup$ – Nick Koprowicz Jul 28 at 21:13
  • $\begingroup$ @NickKoprowicz I use specific strategies to deal with the imbalance class since for my study is more important to predict the failures. Then, if the model predict all "Normal" it will be useless for my analysis. My first idea was to use a machine learning algorithm in order to see if it is capable to predict the exact subperiods in which the failures occur. However, now I don't know if a learning algorithm is really necessary. $\endgroup$ – Fernanda Jul 28 at 21:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.