# Self-designed objective for linear regression learning

A multiple linear regression is to use several predictor variables to predict the outcome of a response variable, like the following relationship:

$y_{i}=\beta_{1}x_{i1}+...+\beta_{p}x_{ip}+\epsilon_{i}, i=1,...,n$

I understand the typical objective to learn the $\beta$ paramters is least-squares, which means to minimize the sum of the sqaure of $\epsilon_{i}$. Now I want other kinds of objective, for example to maximize the Shannon entropy of the sequences of $\epsilon$ (or other self-specified objective). I googled towards this direction but no luck. I am wondering if there is any problem (and tool to solve it if possible) I can look into to do that?

• Does anyone have a clue? – Kimi Jun 29 '15 at 15:01
• This is too vague...are you interested in the Shannon Entropy specifically? – user9424 Jun 29 '15 at 16:34
• @Bey Yes, let's say Shannon Entropy. – Kimi Jun 29 '15 at 20:56
• Ok, well unlike least squares, the Shannon Entropy requires that you be able to assign a probability or density value to each error. Do you want to assume a gaussian error with mean 0 and a pre-specified stdev? Also, do you really want to maximize entropy or minimize entropy? – user9424 Jun 30 '15 at 3:23
• @Bey You are right a pdf is needed. But since we get a sequence of error, i.e. {epsilon_i}, can we use emperical distribution? I think it is the simplist and roughest way but please tell me if I am wrong (like using a Gaussian actually help). And I want to maximize entropy. – Kimi Jun 30 '15 at 13:28