# Multiple linear regression for multi-dimensional input and output?

Assume that I have $$N$$ points $$x_i,i=1,...,N$$ in some $$A>1$$-dimensional space $$\mathbb{R}^A$$ with pointwise evaluations of some function $$f:\mathbb{R}^A \rightarrow \mathbb{R}^B$$, i.e. $$f(x_i),i=1,...,N$$ where $$f(x_i) \in \mathbb{R}^B$$.

It is my goal to find a multiple linear regression between $$x_i$$ and $$f(x_i)$$. Now sklearn has a function (sklearn.linear_model.LinearRegression) for a multiple linear regression for functions of the type $$f:\mathbb{R}^A \rightarrow \mathbb{R}$$, but my output is $$B$$-dimensional. I assume that I could make independent multiple linear regressions for each output dimension and then combine the results, but there must be an easier way of achieving this.

Do you know of a more efficient way?

You are asking about multioutput regression. The class you talked about sklearn.linear_model.LinearRegression supports this out of the box.

import numpy as np
from sklearn.linear_model import LinearRegression

# features
A = 10
# number of values to predict
B = 15
# number of rows in dataset
m = 100

x = np.ones((m, A))
y = np.ones((m, B))

model = LinearRegression()
model.fit(x, y)


sklearn.linear_model.LinearRegression actually just creates B models. However it optimises calculations using vectorisation.

It actually is exactly the same as a fully connected layer in a neural network which has no activation function.

• Well now I feel dumb - I tried doing this before but couldn't get it to work, I must have made some mistake. This is drastically faster than the for-loop solution I have created as a workaround, particularly for large $B$. Thanks a lot! Aug 10, 2020 at 16:34