Assume that I have $N$ points $x_i,i=1,...,N$ in some $A>1$-dimensional space $\mathbb{R}^A$ with pointwise evaluations of some function $f:\mathbb{R}^A \rightarrow \mathbb{R}^B$, i.e. $f(x_i),i=1,...,N$ where $f(x_i) \in \mathbb{R}^B$.

It is my goal to find a multiple linear regression between $x_i$ and $f(x_i)$. Now sklearn has a function (sklearn.linear_model.LinearRegression) for a multiple linear regression for functions of the type $f:\mathbb{R}^A \rightarrow \mathbb{R} $, but my output is $B$-dimensional. I assume that I could make independent multiple linear regressions for each output dimension and then combine the results, but there must be an easier way of achieving this.

Do you know of a more efficient way?


1 Answer 1


You are asking about multioutput regression. The class you talked about sklearn.linear_model.LinearRegression supports this out of the box.

import numpy as np
from sklearn.linear_model import LinearRegression

# features
A = 10
# number of values to predict
B = 15
# number of rows in dataset
m = 100

x = np.ones((m, A))
y = np.ones((m, B))

model = LinearRegression()
model.fit(x, y)

sklearn.linear_model.LinearRegression actually just creates B models. However it optimises calculations using vectorisation.

It actually is exactly the same as a fully connected layer in a neural network which has no activation function.

You can read more about it here: https://machinelearningmastery.com/multi-output-regression-models-with-python/

  • 1
    $\begingroup$ Well now I feel dumb - I tried doing this before but couldn't get it to work, I must have made some mistake. This is drastically faster than the for-loop solution I have created as a workaround, particularly for large $B$. Thanks a lot! $\endgroup$
    – J.Galt
    Commented Aug 10, 2020 at 16:34
  • 1
    $\begingroup$ You can speedup the process using model.fit(x, y, n_jobs=-1). Normally model.fit uses only one the cores from your cpu. If you set n_jobs=-1 it will create as many processes as you have cores and speed up computation. $\endgroup$ Commented Aug 10, 2020 at 16:53
  • $\begingroup$ Oh, in-built parallelization? Awesome, thanks for the tip! $\endgroup$
    – J.Galt
    Commented Aug 10, 2020 at 21:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.