I want to minimise mean square error function to find best alpha value (decay rate) for my model.

Here is the description of my model:

time:              1st_month   2nd_month   3rd_month  4th_month  5th_month
Product_shipped     500            600         453      200        789

 If there is delay in products installed after shipping , we multiply by alpha

 Suppose alpha=-0.01
 We create a lower traingular matrix

 month   p1             p2           p3         p4            p5
  1    495.0249169              
  2     490.0993367 588.119204          
  3     485.2227668 582.2673201 439.6118267     
  4     480.3947196 576.4736635 435.2376159 192.1578878 
  5     475.6147123 570.7376547 430.9069293 190.2458849 750.5200159

M(1,1) is calculated as 500*(e^-alpha*month(=1))

M(2,1) is calculated as 500*(e^-alpha*month(=2))

M(2,2) is calculated as 600*(e^-alpha*month(=2))

So forth and so on.

Then Predicted Product shipment is sum across row:


We have originall Installation:


I want to minimise F(sum(Original_Installation-Predicted_Installation)^2) to find alpha which minimise this. How can we frame this or solve this in Python.


For this kind of problem, I would definitely start with scipy.otpimize methods.

I reproduce here an example on how to use it in your context:

import numpy as np
from scipy.optimize import minimize

ALPHA_TRUE = 0.5 # used only to generate some test data

def model(params, X):
    # here you need to implement your real model
    # for Predicted_Installation
    alpha = params[0]
    y_pred = np.exp(-alpha * X)
    return y_pred

def sum_of_squares(params, X, Y):
    y_pred = model(params, X)
    obj = np.sqrt(((y_pred - Y) ** 2).sum())
    return obj

# generate some test data
X = np.random.random(10) # this is "month" if I understood your problem correctly
Y = model([ALPHA_TRUE], X) # Original_Installation

# perform fit to find optimal parameters
# initial value for alpha (guess)
alpha_0 = 0.1

res = minimize(sum_of_squares, [alpha_0, ], args=(X, Y), tol=1e-3, method="Powell")

The result looks like this:

   direc: array([[-1.12550246e-12]])
     fun: 0.0
 message: 'Optimization terminated successfully.'
    nfev: 225
     nit: 9
  status: 0
 success: True
       x: array(0.5)

You have to take a deep look at the documentation to find the best fitting method depending on whether alpha is bounded or not or whether you have constraints on your parameters.

I have also played around recently with the same kind of stuff using tensorflow gradient descent optimization (example: https://stellasia.github.io/blog/2020-02-29-custom-model-fitting-using-tensorflow/)

  • $\begingroup$ Thank you. Suppose our model has many predictors X1,X2,X3 like pandas dataframe df. Can we pass a dataframe of predictors to model([ALPHA_TRUE], df) The reason I am saying is to arrive at predicted y I might have to use different variables. So does the optimization take that into account? $\endgroup$ – MAC Mar 4 '20 at 18:16
  • $\begingroup$ Example: model([ALPHA_TRUE], X) def model: df[a1]*0.98=k, y=k*exp(X) return y So we are actually passing X a column from data frame but using some other columuns also inside function. So does optimization takes all thiose into consideration? $\endgroup$ – MAC Mar 4 '20 at 18:19
  • $\begingroup$ Yes, you can pass a dataframe or as many arguments as you want to the model function, through the args parameter in scipy.optimize.minimize $\endgroup$ – stellasia Mar 4 '20 at 18:24
  • $\begingroup$ So instead of this model([ALPHA_TRUE], X) I can pass model([ALPHA_TRUE], df) and the whole code remains the same.? $\endgroup$ – MAC Mar 4 '20 at 18:28
  • $\begingroup$ Yes basically it should work the same, if you propagate the dataframe correctly from res = minimize(sum_of_squares, [alpha_0, ], args=(df, Y), tol=1e-3, method="Powell") $\endgroup$ – stellasia Mar 4 '20 at 18:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.