Understanding time series anomaly detection using Autoencoder

Question

I'm studying how to detect anomalies in the time series using an Autoeconder. In particular, I'm following the guide posted in the Keras website, but I don't understand why they are creating and how can I adapt it to my dataset. In their guide they load the dataset and create a sequence:

TIME_STEPS = 288

# Generated training sequences for use in the model.
def create_sequences(values, time_steps=TIME_STEPS):
    output = []
    for i in range(len(values) - time_steps):
        output.append(values[i : (i + time_steps)])
    return np.stack(output)


x_train = create_sequences(df_training_value.values)
print("Training input shape: ", x_train.shape)

The reason why they are using 288 as TIME_STEPS is because they have a value for every 5 mins for 14 days. My questions are:

1. Does this method split the data or is he just creating a 3D variable in the correct format for the Convolutional Autoencoder?

2. I have a dataset where there are stored the measurements of 30 devices. Each device has about 4000 values and it is structured as well:

    device01 0.02;0.13;1.15;0.10;8.30;........;4.20
    device02 0.06;0.13;1.40;0.03;7.40;........;6.30
    ........
    device30 0.03;0.24;1.10;0.43;4.40;........;2.30
   

   Since I do not have a timestamp reference in my dataset, how can I define the TIME_STEPS variable?

Answer 1

The length of TIME_STEPS in one sequence is more of a Hyperparameter. That you should try to optimize.

Does this method split the data or is he just creating a 3D variable in the correct format for the Convolutional Autoencoder?

It simply create dataset for a 1-dimsnional convolutional network.Something like this,

$\hspace{3cm}$

Each row is an instance. Convolution will happen across the row.

I have a dataset where there are stored the measurements of 30 devices

You may simply assume each data point as a Time-step assuming it was collected simultaneously.
But your data has an additional dimension i.e. Devices.

• So you should either build one model for each device or take the average of each device and consider it as the single time-step.
• Or you can apply a 1-D CNN with a 2-D kernel in your first layer, assuming each device as a channel. Check an example here.
• In case, Keras doesn't allow a 2-D kernel, then use a 2D-CNN with kernel size "30xM". With that, the convolution will happen in only one direction.
  
  Beware that if you use the last 2 options, then your pre-processing function will have to change i.e each time-step will have "30x1" as a dimension.