I'm currently implementing a 1D CNN to forecast a time series for an industrial process. Essentially, I give the model 30 time steps (1 time step = 1 minute) of input data captured from 7 different sensors at the process input, and I want it to predict one of the process outputs over the course of the following 15 minutes. Essentially:
data_in.shape = (batch_size, num_channels=7, sequence_len=30) target.shape = (batch_size, num_channels=1, sequence_len=15)
What makes this process a bit tricky to model is that the delay between input/output (i.e. the retention time in the process) is not constant. This means that sometimes what is observed at the output at $t=0$ is most strongly correlated to the input at time $t-15$, but other times it may be most strongly correlated to what happens at time $t-20$. Currently I have my CNN perform a series of padded convolutions and then take the last 15 time steps as the output.
class CNN(nn.Module): def __init__(self, total_dims, batch_size, num_layers, input_steps, output_steps ): super(CNN, self).__init__() self.conv = nn.Sequential().to(device) for layer in range(num_layers): self.conv.append(nn.Conv1d((2 ** layer) * total_dims, (2 ** (layer + 1)) * total_dims, kernel_size=3, stride=1, padding=1)) self.conv.append(nn.BatchNorm1d((2 ** (layer + 1)) * total_dims)) self.conv.append(nn.ReLU()) self.convout = nn.Sequential( nn.Conv1d((2 ** num_layers) * total_dims, 1, kernel_size=3, stride=1, padding=1), nn.BatchNorm1d(1), nn.ReLU(), ).to(device) def forward(self, data_in): out = self.conv(data_in) out = self.convout(out) return out[:, :, -15:]
However, given that the output may not always be most strongly correlated to these last 15 I would like that final crop to be a learnable parameter. Essentially, instead of having that "15" in the final line, I want to replace it with some variable that will change as needed. What is the best way to go about doing this?