0
$\begingroup$

I have a temperature dataset (data every 15 mins) to build a supervised classification/prediction algorithm, but only know one of the true classes (when data is nearly flatlining around 35deg)

However, given the academic literature on the subject and the data visualization it looks like there are three "true" classes: regulated, sub-ideal regulation, and no regulation. Here is a graph with vertical lines delineating 3 potential categories. Graph What is the best practice to build this classification? First attempt binned data every 6hrs and built categories based off stdev, but that seemed a bit arbitrary and did not align well with the three hypothesized categories.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

So, if I understood you correctly, you want to classify your data based on the frequency and amplitude of the temperature change, right?

I think your attempt with stdev is not so far away, but your interval is a little off, 6 hours might be to much. Think about Shannon, your sample rate must be at least twice the highest frequency in your dataset, if you want to reconstruct your signal completely. If your sample rate is 6 hours you might miss the big leaps at the end of the image.

My recommendation would be to use a sliding window and use Fast Fourier Transformation to get to the frequency spectrum. With those datapoints you can try to cluster them with SVM or kNN approaches. I think SVM is the way to go, since your class should be cleanly seperateble. And since you know how many classes you have there is no need for hypertuning this parameter.

I certainly don't know if this is best practice, but I think this would be a feasible approach.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Great input, i will give that a shot and report back. the post was edited to add that the data records are every 15 mins $\endgroup$
    – Evan
    Commented Jan 24, 2018 at 14:36
0
$\begingroup$

Depending on the sampling rate of the data, I'd create a new variable with stdev calculated using a trailing window, and create k-means clusters using stdev.

Based on a visual assessment, you could go with three clusters but try with 2 or 4 too and select whichever results in the smallest WSS(within cluster sum of squares).

Once you've found these you can assign each cluster a class. However, the problem with this strategy is it will only work with future data that has Similar stdev.

Improve this answer
$\endgroup$
3
  • $\begingroup$ The sampling rate is 18mins on average $\endgroup$
    – Evan
    Commented Jan 25, 2018 at 15:20
  • $\begingroup$ Hi Toros91, for k-means, I have my daily st_dev and time stamp. How do you recommend i treat my timestamp to correctly cluster the stdeviations? $\endgroup$
    – Evan
    Commented Mar 12, 2018 at 20:43
  • $\begingroup$ If these are the only two predictors you have, I don't think you need to use the timestamp for k-NN. Just using the stdev should be enough if your goal is to assign a new observation a class. In my original answer, I assumed you had other predictors. In that case, you could use k-means clustering to find the best clusters and then train an SVM or other model using the assigned clusters as the output and the other predictors as inputs. $\endgroup$
    – Murtaza
    Commented Mar 13, 2018 at 23:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.