1
$\begingroup$

I am trying to visualize, how does the linear regression model draw a straight line when we have multiple features and one label to predict. Like when we have 1 feature and 1 label, we can easily plot them on a 2-D graph and draw a straight line. Even if we have two features and one label, we can still plot them on a 3-D graph but I couldn't able to visualize, what will happen when we have 20 features. Will algorithm use 20-D graph or am I missing something.

I am new to machine learning. So this question can come as naive.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

If there are 20 independent variables and 1 dependent variable, a linear regression model can be viewed as a 20-dimensional hyperplane in 21-dimensional space. The hyperplane is not "vertical" with respect to any independent variable. So if you pick a value for each of the 20 independent variables and draw a vertical line at the point consisting of those values, it will cross the hyperplane once at the height that the model outputs when you plug in the values of the independent variables.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Actually I know that we will have a 20-D hyperplane . But for me it is hard to visualize , if you can provide further source where I can find something more about this topic. That'll be a great help. $\endgroup$
    – Ashish M
    Commented Sep 27, 2019 at 21:14
  • 1
    $\begingroup$ You can't visualize it. It's hard enough to visualize 4 dimensions. 21 dimensions is impossible. The previous answer, which disappeared, explained some ways of visualizing high-dimensional data points. But I don't know of any way to directly visualize a high dimensional hyperplane. $\endgroup$
    – Reinstate Monica
    Commented Sep 27, 2019 at 21:44
2
$\begingroup$

Adding to other answers here, I first want to note, that fitting a regression is different to plotting regression results. Fitting simply requires to minimize the sum of squared residuals to determine optimal coefficients (or weights).

A single coefficient gives you the „marginal effect“ of the corresponding variable/feature.

Obviously it is impossible to plot all the results in case we have more than three dimensions (y and two x).

However, you can plot a single effect. You can either just plot the marginal effect, or you can set all remaining variables/features to a specific value, e.g their means. This gives you the opportunity to show the influence of a single variable/feature (given some meaningful values of all remaining features).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.