Using LSTMs for modelling and forecasting several time series generated by the same process

Question

My scenario is roughly the following:

Imagine 500 cars, all Toyota Corollas (or whatever). While these cars have many similarities, they are not exactly identical: some of them have 1.5 liter engines while others have 1.8 liter engines, and some have a high mileage while others have barely left their garage. Still, we can imagine that all cars are the product of one process with a bunch of parameters. Some of these parameters are observed, while others are unobserved.

I have data on the condition of each car at a set of times t_i. For our purposes, "condition" is represented by a number between 0.0 and 1.0, where 0.0 means "as good as new" and 1.0 means "basically a wreck". My goal is to build a model which lets me forecast the condition of each car at some future time, based on its condition at the previous time steps as well as its "parameters" like mileage and engine size. I could train a separate model for each of the cars, but I likely don't have enough data for each car for this model to provide accurate forecasts.

I've already made an attempt at this using ordinary machine learning methods by transforming the full history of each car into one data point by encoding the condition at each of the previous time steps as a feature. My results are kind of so-so, and I'd like to try out LSTMs for this purpose.

However, I have a hard time figuring out how to do this with LSTMs. A bit of googling has led me to lots of tutorials on how to use LSTMs to forecast two properties of the same "object", say, condition and mileage for a single car, but none on how to predict the same property for a bunch of "different objects" with additional parameters. I assume just plugging

[conditionCar1, mileageCar1, colorCar1, conditionCar2, mileageCar2, colorCar2, (...)]

observed at a lot of time steps into an LSTM model wouldn't work very well. Is what I'm trying to do viable, and if so does anybody have any pointers to relevant information?

Answer 1

You can frame the problem this way.

Consider each car as a sample of your dataset and use $Car^{i}$ to denote the $i^{th}$ sample. Then for each sample, you have the following features:

1. Time series of cars' condition $C_0^{i}...C_n^{i}$
2. Other time-invariant features like colour $A^{i}$ and engine size $E^{i}$

(I am not sure how mileage fits here, do you have a time series on this as well? Intuitively mileage changes over the life of a car so it should be time series as well)

My understanding of your question is how to incorporate those time-invariant features into an LSTM model?

Instead of only passing the $C_{t-1}$ as LSTM cell input you pass it with a vector $[C_{t-1}^i, A^i, E^i]$. Here, using Tensorflow's terminology, you input_size is 3 instead 1. $A^i$ and $E^i$ is always the same at each timestep but different across samples.