2
$\begingroup$

I try to predict temperatures values as function of time and different parameters. The temperature curve look like a "ramp" with some "gauss peaks" on regular intervals.

So, I try to build a regression model for the following equation:

$$ \hat{T}_{a, \mu, \sigma, s, c, m, p}\left(t\right) = a \frac{1}{\sigma\sqrt{2 \pi}} e^{-\frac{\left(\sin\left(\frac{t-s}{c}\right) - \mu\right)^2}{2 \sigma^2}} + m t + p $$

My goal is to establish the values of $a$, $\mu$, $\sigma$, $s$, $c$, $m$, $p$.

My first intention was to use the gradient descent, but I'm pretty sure that the derivative of the RSS based cost function won't be convex, so I might get stuck at local mimina.

I'm also wondering if there is no better alternative to solve that problem. It concerns especially methods not involving minimizing derivatives. I think (but not sure) that neuronal networks can help me with that.

So, in short, how would you solve the following?

\begin{equation} \left[\begin{array}{l} a \\ \mu \\ \sigma \\ s \\ c \\ m \\ p \end{array}\right] \leftarrow \textrm{argmin}_{a, \mu, \sigma, s, c, m, p}\frac{1}{2N} \sum_{i=1}^N{\left(a \frac{1}{\sigma\sqrt{2 \pi}} e^{-\frac{\left(\sin\left(\frac{t_i-s}{c}\right) - \mu\right)^2}{2 \sigma^2}} + m t + p - T_i\right)^2} \end{equation}

Thanks a lot for your help.

P.S.: I'm using python 3 with scipy stack

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ The $\frac{1}{2N}$ factor in the cost function is there to make usage of derivatives simpler... $\endgroup$
    – Michael Hooreman
    Commented Dec 15, 2015 at 13:29
  • 1
    $\begingroup$ Neural networks are, for the most part, are trained using gradient descent. Although your problem may not be convex, using gradient descent with a good type of regularization should give you a "good enough" solution. Using global optimizers, such as genetic algorithms, does not guarantee a global optimum will be found, and it will be much more computationally expensive than using gradient descent. Are you strictly against using first order optimization? $\endgroup$
    – Armen Aghajanyan
    Commented Dec 15, 2015 at 19:42
  • 1
    $\begingroup$ Thanks Armen. No, I'm not against first order optimizations ... for the good reason that I don't know what first order means ;-) $\endgroup$
    – Michael Hooreman
    Commented Dec 15, 2015 at 22:07
  • 1
    $\begingroup$ By first order optimization, I mean optimizations that take into account the first derivative. Just a fancy way of saying gradient descent :) $\endgroup$
    – Armen Aghajanyan
    Commented Dec 15, 2015 at 22:59
  • 1
    $\begingroup$ Thanks. Good to know. I think I'll use gradient descent. $\endgroup$
    – Michael Hooreman
    Commented Dec 16, 2015 at 8:38

1 Answer 1

Reset to default
0
$\begingroup$

A loss function that is non-convex can still minimized with gradient descent. Even if the loss function converges to a local mimina, those could be a useful set of parameters.

If you choose to not use gradient descent, scipy has several methods to optimization functions. Try as many as applicable and see which method requires the best set of parameters.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.