I need to classify machine data (sampled by the second) and I need a prediction if a signal is good, medium or bad every ten seconds.
The data that I got from the machine at the moment look like this:
using different linestyles for better visibility. The performance of the machine can be fairly easy indicated by looking at the minimal value and the time it takes to reach this value.
Extending the database
In order to train an algorithm, I extended the database by first doubling it with a random smear effect and then I extended these curves that they reach the minimum later. Hence my extended database is 6 times larger then my start.
In the picture above, the second line denotes the set with which I started. The first red line denotes where something happened (the switching point was changed, so the process took less time and different minimum values were reached) which is not a major problem. I just wanted to mention it for a better understanding of the data.
To achieve one prediction every ten seconds for 200 seconds, I trained 20 models, one taking the first ten seconds, one the first twenty and so on. This worked well, after 30 seconds the confusion matrix shows at lest 94% correct predictions for every class, using a linear SVM. The earlier approaches probably cannot score well due to the data quality.
However, I also tried to make several classifiers which only looked 10, 20, 30 and 50 second into the past, no matter at which position in time. (See here for the algorithm Generate matrix from sliding window on vector) The first entry of the data always corresponded to the position of the frame. The classifiers took a long time to train and and even the 50 seconds classifier performs much worse than the 0-30 seconds classifier. I used linear SVMs here, too, and have not yet tried nonlinear methods.
- Obviously, many small classifiers (about 20) work better than fewer complex ones, and they have the further advantage of increasing the interretability of the result. However, I would like to hear more opinions about this. What are the chances of building a complex classifier with a similar performance and what would be the best way to get there? If a complex classifier would have the same performance, would it still be less desirable?
And, to a lesser extend:
- Would the complex classifier benefit from getting the position from a one-hot-encoder instead of using a number? After all, the position in which the data is at the moment is a crucuial information, however it occupies only one number in the data.
Trying to predict live, but without the benefit of a starting point: How to classify movement data (time series) in real time
Segmentation of data, but with different constraints: Is it possible to implement a classifier according to quarters? What about missing data?
Looking at different aspects of a dataset: How to make machine learning specifically for an individual in a group when we have the data on the group?