Suppose the input time-series feature is $\vec{X}=[\mathbf{x_0},\mathbf{x_1},...\mathbf{x_T}]$, where at each time step $t\in[0,...,T]$, feature $\mathbf{x_t}$ is a vector with dimension $n$. Typical RNN, such as LSTM/GRU, may assume that time-series are sampled with equal interval.

However, in my problem, $\mathbf{x_t}$ is not collected from time interval with equal duration. For example, $\mathbf{x_0}$ is calculated from a time interval with 1min duration. $\mathbf{x_1}$ is calculated from a time interval with 5min duration. It is resonable to expect that $\mathbf{x_1}$ may carray more information than $\mathbf{x_0}$ since it comes from a longer time interval. Thus we want to modified traditional RNN cell such that it can naturally consider this situation.

One of the choice, from my own opinion, is to add an extra deterministc operation in orginal RNN cell. We use GRU as an example here. We modifed the GRU cell as follow:

$$ z = \sigma(\tau_t\odot(U_zx_t) + W_zs_{t-1}) \\ r = \sigma(\tau_t\odot(U_rx_t) + W_rs_{t-1}) \\ h = tanh(\tau_t\odot(U_hx_t) + W_h(s_{t-1}\odot r)) \\ s_t = (1-z)\odot h + z\odot s_{t-1} $$

here $\odot$ is element-wise multiplication and $\tau_t$ is the new added operation. We define $\tau_t = d_t/T$, where $d_t$ is the time duration of feature $\mathbf{x_t}$ at time step $t$ and $T$ is the chosen time scale. Suppose we choose $T=5$min, then $\tau_0=0.2$ and $\tau_1=1.0$ as we described before. Then $\mathbf{x_1}$ will naturally contribute more than $\mathbf{x_0}$ when passing the cell.

Is there any other ideas, expecially published paper that can accomplish the problem given above ? Comments are also wellcome here.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.