Time series: variations as a feature

Question

I am trying to predict clients comportement from market rates.

The value of the products depends on the actual rate but this is not enough. The comportement of the client also depends on their awareness wich depends on the evolution of rates. I've added this in model using past 6 month rates as features in polynomial regression.

In fact media coverage of rate mostly depends on rate variations and I wanted to add that in my model. The idea would be to add a derivative/variation of rate as a feature. But I anticipated something wrong, example with only two month , my variation will be of the form $x_n - x_{n-1}$ that is a simple linear combination of actual and past rates. So for a 1d polynomial regression i will have:

$$ x_{n+1} = a * x_{n} + b * x_{n-1} + c * (x_{n} - x_{n-1})$$

instead of:

$$ x_{n+1} = a_0 * x_{n} + b_0 * x_{n-1}$$

wich is strictly equivalent with $ a + c = a_0 $ and $b-c= b_0$. Higher polynomial degree results in a more or less equivalent result.

I am thinking about a way to include derivative information but it seems not possible. So I am wondering if all the information is included in my curve. Is this a general idea ? all information is somewhat directly contained in data and modifications of features will result in higher order objective function ?

Answer 1

Using derivatives as features is almost the same as using past values, as both reconstruct phase or state space for dynamic system behind the time series. but they differ in some points, like noise amplification and how they carry information. (see Ref: State space reconstruction in the presence of noise; Martin Casdagli; Physica D - 1991 - section 2)

Notice all information is embedded in time series, but using derivatives is going to reinterpret this information, which may be useful or useless.

In your case, if you use all parameters and terms, i believe there is no use in it. but in case of using some algorithms like orthogonal forward regression (OFR) it may be beneficial. (see Ref: Orthogonal least squares methods and their application to non-linear system identification; S. CHEN, S. A. BILLINGS; INT. J. CONTROL, 1989)

Answer 2

The derivative is a linear transform, and you're using a linear model. As you've demonstrated, nothing is gained by adding linear combination of other features (like the derivative) as a new feature in a linear model.

You may want a derivative coefficient to help interpret the model. You could use the normal model $x_{n+1} = a * x_n + b * n_{n-1}$ and then report the value $a-b$ to the client. A more general approach is to examine the frequency response of the model (the derivative is a high-pass filter, your model is a FIR filter).

Alternately you can rotate the basis of the input such that the derivative is a feature and the dimensions are still independent as follows:

You can think of the independent variables $x_n$ and $x_{n-1}$ as forming a standard basis for the two dimensions of your model's input, that is $\langle 1, 0\rangle$ and $\langle 0, 1\rangle$. Neither $a\langle 1, 0 \rangle$ or $b \langle 0, 1 \rangle$ are orthogonal to c$\langle1, -1\rangle$.

Therefore, if you want to use $c(x_n - x_{n-1})$ as a feature, I would also use the orthogonal feature $d(x_n + x_{n-1})$. That is,

$$ x_{n+1} = c(x_n - x_{n-1}) + d(x_n + x_{n-1}) $$.

You could think of this model as the derivative plus the integral.

The same logic applies to the choice of scaling function for the Haar wavelet.