# How to Minimize mean square error using Python

I want to minimise mean square error function to find best alpha value (decay rate) for my model.

Here is the description of my model:

time:              1st_month   2nd_month   3rd_month  4th_month  5th_month
Product_shipped     500            600         453      200        789

If there is delay in products installed after shipping , we multiply by alpha

Suppose alpha=-0.01
We create a lower traingular matrix

month   p1             p2           p3         p4            p5
1    495.0249169
2     490.0993367 588.119204
3     485.2227668 582.2673201 439.6118267
4     480.3947196 576.4736635 435.2376159 192.1578878
5     475.6147123 570.7376547 430.9069293 190.2458849 750.5200159


M(1,1) is calculated as 500*(e^-alpha*month(=1))

M(2,1) is calculated as 500*(e^-alpha*month(=2))

M(2,2) is calculated as 600*(e^-alpha*month(=2))

So forth and so on.

Then Predicted Product shipment is sum across row:

Predicted_Installation
495.0249169
1078.218541
1507.101914
1684.263887
2418.025197


We have originall Installation:

Original_Installation
565
1200
1677
1876
2500


I want to minimise F(sum(Original_Installation-Predicted_Installation)^2) to find alpha which minimise this. How can we frame this or solve this in Python.

For this kind of problem, I would definitely start with scipy.otpimize methods.

I reproduce here an example on how to use it in your context:

import numpy as np
from scipy.optimize import minimize

ALPHA_TRUE = 0.5 # used only to generate some test data

def model(params, X):
# here you need to implement your real model
# for Predicted_Installation
alpha = params[0]
y_pred = np.exp(-alpha * X)
return y_pred

def sum_of_squares(params, X, Y):
y_pred = model(params, X)
obj = np.sqrt(((y_pred - Y) ** 2).sum())
return obj

# generate some test data
X = np.random.random(10) # this is "month" if I understood your problem correctly
Y = model([ALPHA_TRUE], X) # Original_Installation

# perform fit to find optimal parameters
# initial value for alpha (guess)
alpha_0 = 0.1

res = minimize(sum_of_squares, [alpha_0, ], args=(X, Y), tol=1e-3, method="Powell")
print(res)


The result looks like this:

   direc: array([[-1.12550246e-12]])
fun: 0.0
message: 'Optimization terminated successfully.'
nfev: 225
nit: 9
status: 0
success: True
x: array(0.5)


You have to take a deep look at the documentation to find the best fitting method depending on whether alpha is bounded or not or whether you have constraints on your parameters.

I have also played around recently with the same kind of stuff using tensorflow gradient descent optimization (example: https://stellasia.github.io/blog/2020-02-29-custom-model-fitting-using-tensorflow/)

• Thank you. Suppose our model has many predictors X1,X2,X3 like pandas dataframe df. Can we pass a dataframe of predictors to model([ALPHA_TRUE], df) The reason I am saying is to arrive at predicted y I might have to use different variables. So does the optimization take that into account? – MAC Mar 4 '20 at 18:16
• Example: model([ALPHA_TRUE], X) def model: df[a1]*0.98=k, y=k*exp(X) return y  So we are actually passing X a column from data frame but using some other columuns also inside function. So does optimization takes all thiose into consideration? – MAC Mar 4 '20 at 18:19
• Yes, you can pass a dataframe or as many arguments as you want to the model function, through the args parameter in scipy.optimize.minimize – stellasia Mar 4 '20 at 18:24
• So instead of this model([ALPHA_TRUE], X) I can pass model([ALPHA_TRUE], df) and the whole code remains the same.? – MAC Mar 4 '20 at 18:28
• Yes basically it should work the same, if you propagate the dataframe correctly from res = minimize(sum_of_squares, [alpha_0, ], args=(df, Y), tol=1e-3, method="Powell") – stellasia Mar 4 '20 at 18:30