1
$\begingroup$

I have chemical plant data where the product is manufactured in batches. Each batch takes about 4 hours and I have data for every 5 mins. My objective is to classify the batches as good and bad. How can I achieve this for a time related data like this one?

One method I have read is time unfolding where each time interval in combination with the feature becomes an individual feature (e:g if there is interval T0 and T1, features F0 and F1 then we will have 4 features T0 F0 and T1 F1) - please correct me if I am wrong about unfolding.

So basically we are converting each batch of N rows of data into single row. Is there any other method for this?

Improve this question
$\endgroup$
1
  • $\begingroup$ Hi! You can look at my paper ceur-ws.org/Vol-2322/dsi4-1.pdf - section Related Work. There are a few approaches and software libraries to feature extraction from time-series data. $\endgroup$ – wind Apr 29 '19 at 8:46
1
$\begingroup$

I am not clear on how you convert your N rows into a single row. Do you concatenate all of them? This is an option but could become problematic over longer sequence lengths.

A straightforward method could be to simply average the features across time-steps. Although you lose causal information, this has been shown to work surprisingly well for word embeddings.

Another standard approach is to use hidden markov models. Very simply, a hidden markov model assumes a latent state space which generates an output (in this case your feature) at every time-step conditioned on the previous state. https://en.wikipedia.org/wiki/Hidden_Markov_model

Today the practice is to use sequence neural models such as LSTMs and GRUs which can hold memory over longer time-steps. https://en.wikipedia.org/wiki/Long_short-term_memory

I suggest your read up on these methods and select the one that suits your purpose :)

Improve this answer
$\endgroup$
1
  • $\begingroup$ -Hi, here is an example for converting N rows into one row. Lets say there are 5 features F0-F4 and 10 time intervals T0-T9. Since data changes at every time interval, we can consider each time interval in combination with a feature to be a new feature. so 10 rows can be converted into 10*5 features(columns) and one single row (each feature in combination with each time interval), hope this makes sense. Taking an average is a good idea but as you pointed out, we will loose lot of data, I will have a look at Markov, thanks. Any other idea? $\endgroup$ – KarthikKulkarni May 6 '19 at 10:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.