I'm looking for a Python package that implements multivariate linear regression.

(Terminological note: multivariate regression deals with the case where there are more than one dependent variables while multiple regression deals with the case where there is one dependent variable but more than one independent variables.)

  • $\begingroup$ I'm also interested in this but want just the feature vector after the non-linear transform. So on row would be $[1,x_1,x_2,x_1x_2,x_1^2,x_2^2]$ say for degree 2 model with 2 variables. $\endgroup$ Commented Aug 28, 2017 at 22:38

2 Answers 2


You can still use sklearn.linear_model.LinearRegression. Simply make the output y a matrix with as many columns as you have dependent variables. If you want something non-linear, you can try different basis functions, use polynomial features, or use a different method for regression (like a NN).

  • 1
    $\begingroup$ Are you asking specifically about multivariate logistic regression? As in you want to perform many classifications at once? Multivariate linear regression is certainly implemented. Logistic regression would have to be framed differently to use the sklearn library. $\endgroup$
    – jamesmf
    Commented Oct 29, 2015 at 18:34
  • $\begingroup$ Whoops, sorry I misread, I was reading the sklearn.linear_model.LogisticRegression documentation thinking about linear regression. I'll remove my comment to avoid confusing future readers. Thanks! $\endgroup$ Commented Oct 30, 2015 at 3:30
  • $\begingroup$ wish you would have emphasized how to get the polynomial feature vector... $\endgroup$ Commented Aug 28, 2017 at 23:02
  • $\begingroup$ scikit-learn.org/stable/modules/generated/… $\endgroup$
    – jamesmf
    Commented Aug 29, 2017 at 1:15

Just for fun you can compute the feature by hand by forming tuples $seq =(d_1,...,d_N)$ such that $Sum(seq) = \sum^N_{i=1} \leq D$. Once you form those tuples each entry indicates the power the current raw feature should be raised by. So say $(1,2,3)$ would map to the monomial $x_1 x_2^2 x_3^3$.

The code to get the tuples is:

def generate_all_tuples_for_monomials(N,D):
    if D == 0:
        seq0 = N*[0]
        sequences_degree_0 = [seq0]
        S_0 = {0:sequences_degree_0}
        return S_0
        # S_all = [ k->S_D ] ~ [ k->[seq0,...,seqK]]
        S_all = generate_all_tuples_for_monomials(N,D-1)# S^* = (S^*_D-1) U S_D
        S_D_current = []
        # for every prev set of degree tuples
        #for d in range(len(S_all.items())): # d \in [0,...,D_current]
        d = D-1
        d_new = D - d # get new valid degree number
        # for each sequences, create the new valid degree tuple
        S_all_seq_for_deg_d = S_all[d]
        for seq in S_all[d]:
            for pos in range(N):
                seq_new = seq[:]
                seq_new[pos] = seq_new[pos] + d_new # seq elements dd to D
                if seq_new not in S_D_current:
        S_all[D] = S_D_current
        return S_all

then it should be easy to do regression if you know linear algebra.

c = pseudo_inverse(X_poly)*y

example. Probably better to do regularized linear regression though if your interested in generalization.

Acknowledgements to Yuval is CS exchange for the help.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.