# 1st order Taylor Series derivative calculation for autoregressive model

I wrote a blog post where I calculated the Taylor Series of an autoregressive function. It is not strictly the Taylor Series, but some variant (I guess). I'm mostly concerned about whether the derivatives look okay. I noticed I made a mistake and fixed the issue. It seemed simple enough,but after finding an error, I started to doubt myself.

$$f(t+1) = w_{t+1} \cdot f(t)$$

$$y^{*}_{t+1} = f(t+1)-{\frac {f'(t+1)}{1!}}(-t-1+t)$$

$$y^{*}_{t+1} = w_{t+1} f(t) + \dfrac{d}{df(t)}w_{t+1}f(t) + \dfrac{d}{dw_{t+1}}w_{t+1}f(t)$$

$$y'_{t+1} = w_{t+1} f(t) + w_{t+1} + f(t)$$

The details can be found in the blog post:

EDIT 7/6/20:

The AR form:

$$y^{*}_{t+1}=c+\sum _{{I=0}}^{L}w _{t+1-i}y_{{t-i}}+\varepsilon _{t}$$

f(t) is a recursive dense layer, y is the predicted output, and w are the weights, and L are the number of lag components. For the simple case where the next value only depends on the previous value, I got the following result.

$$f(t+1) = w_{t+1} \cdot f(t)$$

$$y^{*}_{t+1} = f(t+1)-{\frac {f'(t+1)}{1!}}(-t-1+t)$$

$$y^{*}_{t+1} = w_{t+1} f(t) + \dfrac{d}{df(t)}w_{t+1}f(t)$$

$$y'_{t+1} = w_{t+1} f(t) + w_{t+1}$$

EDIT 7/7/20:

The function f(t) represents y(t) with an error term. The error term might have some random process, but I'm going to assume that the errors are independent.

$$f(t+1) = w_{t+1} \cdot y(t) + \epsilon_t$$

EDIT 7/9/20:

Changed the dimensionality of w_t+1 to w_t.

$$f(t+1) = w_{t} \cdot f(t)$$

$$y^{*}_{t+1} = f(t+1)-{\frac {f'(t+1)}{1!}}(-t-1+t)$$

$$y^{*}_{t+1} = w_{t} f(t) + \dfrac{d}{df(t)}w_{t}f(t)$$

$$y'_{t+1} = w_{t} f(t) + w_{t}$$

• I am not sure I understand the post, nor why the chain rule (or perhaps you mean the product rule) for derivatives would result in the 3rd equation... – Nikos M. Jul 5 '20 at 16:46
• Oh you are right, it is actually the partial derivative + the product rule. – FreedomToWin Jul 5 '20 at 21:02
• The purpose was to linearize the recursive formula using the Taylor Series approximation. I couldn't find any resources on how to do this so I took a stab at it. – FreedomToWin Jul 6 '20 at 1:30
• Where is the actual AR (auto-regressive) or ARMA (auto-regressive-moving average) model? Where are these equations. it seems you start directly from trying to find the adaptive rule for the weights. For example what is $f(t)$? I presume $y_t$ is the output and $w_t$ is the weight. But where is the actual model? Please update your question with the model equation first, then try to find the adaptive rule for weights – Nikos M. Jul 6 '20 at 9:06
• OK but I dont make much sense of this definition of $f(t)$. Why does it multiply $y_t$ with $w_{t+1}$ (doesnt make much sense, dimensional analysis, for example, fails)? You already have an error term $\epsilon_t$ in the definition of the AR model. Why is this function needed, what purpose does it fit? – Nikos M. Jul 9 '20 at 6:33

$$y^{*}_{t+1} = w_{t+1}f(t) + \frac{df(t+1)}{d(t+1)}$$
$$y^{*}_{t+1} = w_{t+1}f(t) + \frac{d(w_{t+1}f(t))}{dt}*\frac{dt}{d(t+1)}$$
$$y^{*}_{t+1} = w_{t+1}f(t) + w_{t+1}\frac{df(t)}{dt} + f(t)\frac{dw_{t+1}}{dt}$$