# Regression problem - too complex for gradient descent

I try to predict temperatures values as function of time and different parameters. The temperature curve look like a "ramp" with some "gauss peaks" on regular intervals.

So, I try to build a regression model for the following equation:

$$\hat{T}_{a, \mu, \sigma, s, c, m, p}\left(t\right) = a \frac{1}{\sigma\sqrt{2 \pi}} e^{-\frac{\left(\sin\left(\frac{t-s}{c}\right) - \mu\right)^2}{2 \sigma^2}} + m t + p$$

My goal is to establish the values of $a$, $\mu$, $\sigma$, $s$, $c$, $m$, $p$.

My first intention was to use the gradient descent, but I'm pretty sure that the derivative of the RSS based cost function won't be convex, so I might get stuck at local mimina.

I'm also wondering if there is no better alternative to solve that problem. It concerns especially methods not involving minimizing derivatives. I think (but not sure) that neuronal networks can help me with that.

So, in short, how would you solve the following?

\begin{equation} \left[\begin{array}{l} a \\ \mu \\ \sigma \\ s \\ c \\ m \\ p \end{array}\right] \leftarrow \textrm{argmin}_{a, \mu, \sigma, s, c, m, p}\frac{1}{2N} \sum_{i=1}^N{\left(a \frac{1}{\sigma\sqrt{2 \pi}} e^{-\frac{\left(\sin\left(\frac{t_i-s}{c}\right) - \mu\right)^2}{2 \sigma^2}} + m t + p - T_i\right)^2} \end{equation}

Thanks a lot for your help.

P.S.: I'm using python 3 with scipy stack

• The $\frac{1}{2N}$ factor in the cost function is there to make usage of derivatives simpler... Dec 15, 2015 at 13:29
• Neural networks are, for the most part, are trained using gradient descent. Although your problem may not be convex, using gradient descent with a good type of regularization should give you a "good enough" solution. Using global optimizers, such as genetic algorithms, does not guarantee a global optimum will be found, and it will be much more computationally expensive than using gradient descent. Are you strictly against using first order optimization? Dec 15, 2015 at 19:42
• Thanks Armen. No, I'm not against first order optimizations ... for the good reason that I don't know what first order means ;-) Dec 15, 2015 at 22:07
• By first order optimization, I mean optimizations that take into account the first derivative. Just a fancy way of saying gradient descent :) Dec 15, 2015 at 22:59
• Thanks. Good to know. I think I'll use gradient descent. Dec 16, 2015 at 8:38