Questions tagged [linear-regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
6 votes
3 answers
1k views

Correlation vs Multicollinearity

I have been taught to check correlation matrix before going for any algorithm. I have a few questions around the same: Pearson Correlation is for numerical variables only. What if we have to check ...
6 votes
2 answers
441 views

Confidence interval interpretation in linear regression when errors are not normally distributed

I've read that "If the error distribution is significantly non-normal, confidence intervals may be too wide or too narrow" (source). So, can anyone elaborate on this? When are the confidence intervals ...
9 votes
1 answer
983 views

Assumptions of linear regression

In simple terms, what are the assumptions of Linear Regression? I just want to know that when I can apply a linear regression model to our dataset.
3 votes
1 answer
5k views

How to interpret Variance Inflation Factor (VIF) results?

From various books and blog posts, I understood that the Variance Inflation Factor (VIF) is used to calculate collinearity. They say that VIF till 10 is good. But I have a question. As we can see in ...
1 vote
2 answers
2k views

Finding optimal weights for models

I'm trying to implement an algorithm to find the minimal value of a function. Before moving to sigmoid activation functions, i'm trying to understand linear regression. Usually, a gradient descent ...
  • 389
10 votes
2 answers
5k views

Linear Regression and scaling of data

The following plot shows coefficients obtained with linear regression (with mpg as the target variable and all others as predictors). For mtcars dataset (here and ...
  • 1,456
6 votes
3 answers
2k views

xgboost: Is there a way to perform regression on rates/percentages data?

I have a dependent variable, $Y$, that is made up of rates/percentages data, so each value is between $0$ and $1$. I was attracted to the xgboost library because it allows focusing in on specific ...
109 votes
5 answers
66k views

Why do cost functions use the square error?

I'm just getting started with some machine learning, and until now I have been dealing with linear regression over one variable. I have learnt that there is a hypothesis, which is: $h_\theta(x)=\...
  • 1,263
45 votes
5 answers
41k views

How to force weights to be non-negative in Linear regression

I am using a standard linear regression using scikit-learn in python. However, I would like to force the weights to be all non-negative for every feature. is there any way I can accomplish that? I was ...
  • 1,891
11 votes
3 answers
21k views

Can GPS coordinates (latitude and longitude) be used as features in a linear model?

I have data sets that contain, among many features, GPS coordinates (latitude and longitude). I'd like to use these data sets to explore problems such as: (1) computing ETA to drive between start and ...
3 votes
2 answers
5k views

How to reduce the error in linear regression [closed]

In linear regression, how can we minimize the error term?
  • 534
7 votes
1 answer
2k views

Why Scikit and statsmodel provide different Coefficient of determination?

First of all, I know there is a similar question, however, I didn't find it so much helpful. My issue is concerning simple Linear regression and the outcome of R-Squared. I founded that results can ...
3 votes
1 answer
225 views

For a linear model without intercept, why does the redundent term in one-hot encoding function as intercept?

In this question Elias Strehle pointed out that if we keep all the levels during one hot encoding on a linear model without an intercept, the redundant feature will function as an intercept. Why is ...
1 vote
1 answer
79 views

How to use a a trained model

I just trained my first model in Python 3.7/scikitlearn (Linear Regression) (well I copied most of the code but its something ^^). Now I want to actually Use the model. Specifically its about sons ...
  • 181
16 votes
4 answers
44k views

What does "linear in parameters" mean?

The model of linear regression is linear in parameters. What does this actually mean?
10 votes
3 answers
6k views

Is there a library that would perform segmented linear regression in python?

There is a package named segmented in R. Is there a similar package in python?
8 votes
2 answers
7k views

Is it valid to shuffle time-series data for a prediction task?

I have a time-series dataset that records some participants' daily features from wearable sensors and their daily mood status. The goal is to use one day's daily features and predict the next day's ...
  • 83
13 votes
2 answers
5k views

Why using L1 regularization over L2?

Conducting a linear regression model using a loss function, why should I use $L_1$ instead of $L_2$ regularization? Is it better at preventing overfitting? Is it deterministic (so always a unique ...
8 votes
2 answers
14k views

What is the Time Complexity of Linear Regression?

I am working with linear regression and I would like to know the Time complexity in big-O notation. The cost function of linear regression without an optimisation algorithm (such as Gradient descent) ...
9 votes
1 answer
13k views

Implementation of Stochastic Gradient Descent in Python

I am attempting to implement a basic Stochastic Gradient Descent algorithm for a 2-d linear regression in Python. I was given ...
  • 203
8 votes
2 answers
199 views

Time-series prediction: Model & data assumptions in AI/ML models vs conventional models

I was wondering if there was a good paper out there that informs about model and data assumptions in AI/ML approaches. For example, if you look at Time Series Modelling (Estimation or Prediction) ...
  • 538
6 votes
1 answer
2k views

Why after adding categorical data the Linear Regression fails?

Based on a training set we applied a simple Linear Regression on some attributes that all were numeric. Now we have more attributes in terms of categories and of course we applied one-hot-encoding to ...
4 votes
2 answers
1k views

Interpreting the evaluation result of multiple linear regression

I am learning the multiple linear regression model. I've built a model and using R command: summary(model) I got this result: ...
  • 181
8 votes
3 answers
3k views

Is a "curve" considered "linear"?

In linear regression, we are fitting a polynomial to a set of data points. In Bishop's book of Pattern Recognition & Machine Learning, there are a few examples where the fit is a curve or a ...
  • 469
7 votes
2 answers
728 views

Why augmenting the training data with binary attributes works better for our dataset?

We have a dataset with multiple features ~400 where all of the features have a histogram as you can see in the following picture (sampled only a few) Our assumption We thought that this looked like ...
6 votes
2 answers
6k views

Is NN with no hidden layer is behave like a regression?

Is a NN with no hidden layer is behave like a regression? What we could say that NN without hidden layer can say us? ​ If we have for instance 20 input and 4 output and I have no true label, is it ...
  • 1,636
0 votes
2 answers
201 views

how Lasso regression helps to shrinks the coefficient to zero and why ridge regression dose not shrink the coefficient to zero?

How does Lasso regression help with feature selection of model by making the coefficient shrink to zero? I could see few below with below diagram. Can any please explain in simple terms how to ...
  • 1,261
7 votes
3 answers
12k views

Difference between Ridge and Linear Regression

From what I have understood, the Ridge Regression is just having the loss function for an optimization problem with the addition of the regularization term (L2 Norm in the case of Ridge). However I am ...
7 votes
2 answers
866 views

Can a linear regression model without polynomial features overfit?

I've read in some articles on the internet that linear regression can overfit. However is that possible when we are not using polynomial features? We are just plotting a line trough the data points ...
3 votes
1 answer
9k views

Plotting multivariate linear regression

For practicing linear regression, I am generating some synthetic data samples as follows. First it generates 2000 samples with 3 features (represented by x_data). ...
2 votes
1 answer
636 views

Predicting house price using linear regression [closed]

I'm trying to predict a house price using linear regression method. I gather the real data from a real estate website. I have some features and two numerical value in which the price is the target ...
  • 141
2 votes
1 answer
855 views

Can GLM( generalized linear method) handle the collinearity between the predictor variables in a regression-analysis?

I'm a beginner in Machine learning and I've studied that collinearity among the predictor variables of a model is a huge problem since it can lead to unpredictable model behaviour and a large error. ...
  • 157
1 vote
2 answers
271 views

How does "linear algebraic" weight training function work?

This answer shows that linear and polynomial function weights can be trained using this matrix operation: $w = (X^TX)^{-1}X^Ty$ Therefore, algorithms such as gradient descent are not necessary for ...
  • 389
9 votes
1 answer
2k views

Implementing simple linear regression using a neural network

I have been trying to implement simple linear regression using neural networks in Keras in hope of understanding how to work in the Keras library. Unfortunately, I am ending up with a very bad model. ...
3 votes
2 answers
2k views

Reason for generally using RMSE instead of MSE in Linear Regression

In linear regression, why we generally use RMSE instead of MSE? The rationale I know is that it's easy to minimize the error in RMSE instead of MSE by Gradient Descent, but I need to know the exact ...
  • 534
2 votes
4 answers
2k views

How do you predict a continuous variable when all your independent variables are categorical

I am new to data science and ML. Recently I have been given a sales dataset which contains weekly sales of a fashion brand. It has information about the product like category(t shirt, polo shirt, ...
  • 21
1 vote
3 answers
1k views

R-square or adjusted R-square for one variable model?

I have model like y=mx. Since the adjusted R2 tells you the percentage of variation explained by only the independent variables that actually affect the dependent variable and I have only one ...
1 vote
3 answers
288 views

Why would it be bad to fit a regression model to a binary classification problem?

Let's say that we have a binary classification problem. Why would it be bad to fit a linear regression and then classify given a threshold? The output would be continuos and it could be out of range,...
0 votes
2 answers
131 views

Predicting the likelihood that a prediction from a linear regression model is accurate

So to set up the problem: I have a data set that had labeled data like colour, brand and quality as independent variables and the dependent is RRP (price). I have made a linear regression model using ...
  • 115