3
$\begingroup$

I'm new to RNNs and LSTM and would like some direction with a problem I have. I have a data set containing system metrics (like CPU utilization, disk operations, memory use) of an AWS EC2 instance with a total of 7 columns and around 8000 rows. Each row also has a timestamp with a 5 min interval between each row.

I want to build a LSTM model to forecast the performance for let's say the next hour based on my data. What would be the best approach for solving a problem like this? I know this can be done in many different ways but I would really appreciate some input how to go about this.

$\endgroup$
  • 1
    $\begingroup$ This is the result I got from following the tutorial on machine learning mastery. Any help interpreting the result? i.stack.imgur.com/TRGGx.jpg $\endgroup$ – Andpej Mar 14 at 7:55
1
$\begingroup$

LSTM should be a good fit for this problem(as you mentioned in question). It can identify patterns in sequence of data (For example : high memory usage is a leading indicator of higher disk usage [due to higher usage of swap partition]).

This is a tutorial for similar use-case :

https://machinelearningmastery.com/multivariate-time-series-forecasting-lstms-keras/

$\endgroup$
  • $\begingroup$ Thank you, I will check the tutorial. Since every row in my dataset represent a 5 minute period, do you think it would be beneficial for my result to concatenate them into hours, or even days? If i keep it in the format I already have, should every row in the dataset represent one timestep? Sorry about all the questions, but I try to wrap my head around how to think when developing a LSTM model. $\endgroup$ – Andreas Mar 9 at 10:02
  • $\begingroup$ Yes, create a set of rows and create single training example. This example can represent 1 day (~300 rows) or 1 Hour . Try both to see which set is large enough for LSTM to learn repetitive patterns. $\endgroup$ – Shamit Verma Mar 9 at 16:24
  • $\begingroup$ This is the result I got from following the tutorial on machine learning mastery. Any help interpreting the result? i.stack.imgur.com/TRGGx.jpg $\endgroup$ – Andreas Mar 14 at 8:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.