I have a multivariate time series of driving scenarios which has X,Y positions, speed, orientation etc. of the vehicles. Each scenario A, B, C, D etc. are of different lengths with different delta ts for each scenario. I would like to use LSTM autoencoder to do dimensionality reduction so that I can later on use maybe clustering or other tasks on the representation. I came across that this the good process to go: data normalization using standard scaler--> Use sliding window approach to creater inputs as smaller sequences for the model (not sure if this highly beneficial)--> Padding and Masking --> Build and Train LSTM Autoencoder. I am new to ML but I do see super high losse. I am starting to doubt my process of sliding window or anything to do with pre-processing before I go forward with refining my model.

What do I expect to see from clusters?

I want to see homogenous scenario types like right-turn, left-turn, UTurn and standstill cluster groups:

What have I tried before? -DTW only on the X,Y positions —> PCA —> K means Clustering: I am still unsure if this is good enough approach. I am not sure if DTW captures temporal dependencies between the trajectories so that it knows scenario types.

Why autoencoder?

I want to see if reducing dimensions (probably to 2/3 dimensions) of my multi variate time series using an LSTM autoencoder can help in clustering in unsupervised way.

This is my approach to dimensionality reduction using autoencoder and my modeling part below. I would like to if my approach to padding the variable length trajectories and then masking and then creating model is good start? I see the losses are too high:

import numpy as np
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, LSTM, RepeatVector, TimeDistributed, Masking, Dense
from tensorflow.keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.callbacks import EarlyStopping
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt

grouped = filtered_df.groupby('name')
trajectories = {name: group[['time', 'positionX', 'positionY', 'Orientation', 
'Speed']].dropna().to_numpy() for name, group in grouped}

# Extract and scale trajectory features (excluding time)
scaled_trajectories = []
for traj in trajectories.values():
   scaler = StandardScaler()
   scaled_features = scaler.fit_transform(traj[:, 1:])  # Exclude the first column (time)

# Define constants
special_value = -755.0
max_seq_len = max(len(t) for t in scaled_trajectories)
num_features = scaled_trajectories[0].shape[1]  # Number of features excluding time

# Pad the sequences
X_padded = pad_sequences(scaled_trajectories, maxlen=max_seq_len, padding='post', 
dtype='float32', value=special_value)

# Define the LSTM Autoencoder
latent_dim = 3 #or set to 2
encoder_inputs = Input(shape=(max_seq_len, num_features))
masked_input = Masking(mask_value=special_value)(encoder_inputs)
encoded = LSTM(latent_dim, return_sequences=False)(masked_input)
decoded = RepeatVector(max_seq_len)(encoded)
decoded = LSTM(num_features, return_sequences=True)(decoded)

autoencoder = Model(encoder_inputs, decoded)
autoencoder.compile(optimizer='adam', loss='mse')

# Early stopping callback
early_stopping = EarlyStopping(monitor='val_loss', patience=10, restore_best_weights=True)

# Fit the model with early stopping
history = autoencoder.fit(X_padded, X_padded, epochs=100, batch_size=10, validation_split=0.2, 

# Plotting the training and validation loss
plt.plot(history.history['loss'], label='Training Loss')
plt.plot(history.history['val_loss'], label='Validation Loss')
plt.title('Model Loss During Training')

# Create a model for the encoder part
encoder_model = Model(encoder_inputs, encoded)

# Dimensionality reduction
latent_representations = encoder_model.predict(X_padded)
# latent_representations now holds the compressed form of your trajectories
print("Shape of latent representations:", latent_representations.shape)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.