Questions tagged [linear-regression]

Techniques for analyzing the relationship between one (or more) "dependent" variables and "independent" variables.

Filter by
Sorted by
Tagged with
115 votes
5 answers
70k views

Why do cost functions use the square error?

I'm just getting started with some machine learning, and until now I have been dealing with linear regression over one variable. I have learnt that there is a hypothesis, which is: $h_\theta(x)=\...
Golo Roden's user avatar
  • 1,323
46 votes
5 answers
48k views

How to force weights to be non-negative in Linear regression

I am using a standard linear regression using scikit-learn in python. However, I would like to force the weights to be all non-negative for every feature. is there any way I can accomplish that? I was ...
user's user avatar
  • 1,993
32 votes
3 answers
137k views

How can I check the correlation between features and target variable?

I am trying to build a Regression model and I am looking for a way to check whether there's any correlation between features and target variables? This is my ...
Jeeth's user avatar
  • 931
27 votes
5 answers
39k views

Python library for segmented regression (a.k.a. piecewise regression)

I am looking for a Python library that can perform segmented regression (a.k.a. piecewise regression). Example:
Franck Dernoncourt's user avatar
20 votes
3 answers
18k views

Overfitting in Linear Regression

I'm just getting started with machine learning and I have trouble understanding how overfitting can happen in a linear regression model. Considering we use only 2 feature variables to train a model, ...
Sachin Krishna's user avatar
17 votes
4 answers
52k views

What does "linear in parameters" mean?

The model of linear regression is linear in parameters. What does this actually mean?
Albert Gao's user avatar
16 votes
2 answers
78k views

When to choose linear regression or Decision Tree or Random Forest regression? [closed]

I am working on a project and I am having difficulty in deciding which algorithm to choose for regression. I want to know under what conditions should one choose a <...
Jason Donnald's user avatar
13 votes
1 answer
57k views

How to do stepwise regression using sklearn? [duplicate]

I could not find a way to stepwise regression in scikit learn. I have checked all other posts on Stack Exchange on this topic. Answers to all of them suggests using f_regression. But f_regression ...
nlahri's user avatar
  • 131
13 votes
2 answers
5k views

Why using L1 regularization over L2?

Conducting a linear regression model using a loss function, why should I use $L_1$ instead of $L_2$ regularization? Is it better at preventing overfitting? Is it deterministic (so always a unique ...
astudentofmaths's user avatar
12 votes
7 answers
2k views

Multi-country model or single model

I am working on a ML model to be deployed in a product operating in many countries. The issue that I am having is the following: should I train one model and serve it for all countries? train a model ...
David Masip's user avatar
  • 6,051
12 votes
1 answer
14k views

XGBoost Linear Regression output incorrect

I am a newbie to XGBoost so pardon my ignorance. Here is the python code : ...
simplfuzz's user avatar
  • 223
11 votes
2 answers
18k views

What is the Time Complexity of Linear Regression?

I am working with linear regression and I would like to know the Time complexity in big-O notation. The cost function of linear regression without an optimisation algorithm (such as Gradient descent) ...
user134132523's user avatar
11 votes
3 answers
25k views

Can GPS coordinates (latitude and longitude) be used as features in a linear model?

I have data sets that contain, among many features, GPS coordinates (latitude and longitude). I'd like to use these data sets to explore problems such as: (1) computing ETA to drive between start and ...
stackoverflowuser2010's user avatar
10 votes
1 answer
2k views

Assumptions of linear regression

In simple terms, what are the assumptions of Linear Regression? I just want to know that when I can apply a linear regression model to our dataset.
Anvay Joshi's user avatar
10 votes
3 answers
7k views

Is there a library that would perform segmented linear regression in python?

There is a package named segmented in R. Is there a similar package in python?
vikasreddy's user avatar
10 votes
2 answers
7k views

Linear Regression and scaling of data

The following plot shows coefficients obtained with linear regression (with mpg as the target variable and all others as predictors). For mtcars dataset (here and ...
rnso's user avatar
  • 1,578
10 votes
2 answers
10k views

Is it valid to shuffle time-series data for a prediction task?

I have a time-series dataset that records some participants' daily features from wearable sensors and their daily mood status. The goal is to use one day's daily features and predict the next day's ...
Han's user avatar
  • 103
9 votes
3 answers
16k views

What is the difference between residual sum of squares and ordinary least squares?

They look like the same thing to me but I'm not sure. Update: in retrospect, this was not a very good question. OLS refers to fitting a line to data and RSS is the cost function that OLS uses. It ...
sebastianspiegel's user avatar
9 votes
2 answers
6k views

How to normalize data without knowing the min and max values?

I have a Lending club dataset from Kaggle; it contains many different columns: there are for example dummy variables, years, amount of the loan...ect I want to normalize the data in the training and ...
Ghassen Ben Hamida's user avatar
9 votes
1 answer
3k views

Implementing simple linear regression using a neural network

I have been trying to implement simple linear regression using neural networks in Keras in hope of understanding how to work in the Keras library. Unfortunately, I am ending up with a very bad model. ...
mathisbetter's user avatar
9 votes
1 answer
13k views

Implementation of Stochastic Gradient Descent in Python

I am attempting to implement a basic Stochastic Gradient Descent algorithm for a 2-d linear regression in Python. I was given ...
foobarbaz's user avatar
  • 203
9 votes
1 answer
5k views

feature importance via random forest and linear regression are different

Applied Lasso to rank the features and got the following results: ...
neurite's user avatar
  • 193
9 votes
2 answers
5k views

Dealing with feature vectors of variable length

How does one deal with a feature vector that can vary in size? Let's say per object, I calculate 4 features. In order to solve a certain regression problem, I may have 1, 2, or more of these objects (...
Otto Nahmee's user avatar
8 votes
5 answers
2k views

What does it mean when people say a cost function is something you want to minimize?

I am having a lot of trouble understanding this. Does it mean you should not use the cost function very often?
jame_smith's user avatar
8 votes
2 answers
4k views

Why Scikit and statsmodel provide different Coefficient of determination?

First of all, I know there is a similar question, however, I didn't find it so much helpful. My issue is concerning simple Linear regression and the outcome of R-Squared. I founded that results can ...
Luckasino's user avatar
  • 183
8 votes
3 answers
18k views

Difference between Ridge and Linear Regression

From what I have understood, the Ridge Regression is just having the loss function for an optimization problem with the addition of the regularization term (L2 Norm in the case of Ridge). However I am ...
Panathinaikos's user avatar
8 votes
3 answers
4k views

Is a "curve" considered "linear"?

In linear regression, we are fitting a polynomial to a set of data points. In Bishop's book of Pattern Recognition & Machine Learning, there are a few examples where the fit is a curve or a ...
Srishti M's user avatar
  • 471
8 votes
2 answers
21k views

Obtaining a confidence interval for the prediction of a linear regression

The data I am working with is being used to predict the duration of a trip between two points. There are about 100 different trips in the data and ~90k observations. I am using the standard pattern: ...
ericg's user avatar
  • 181
8 votes
2 answers
1k views

Can a linear regression model without polynomial features overfit?

I've read in some articles on the internet that linear regression can overfit. However is that possible when we are not using polynomial features? We are just plotting a line trough the data points ...
Tim von Känel's user avatar
8 votes
2 answers
231 views

Time-series prediction: Model & data assumptions in AI/ML models vs conventional models

I was wondering if there was a good paper out there that informs about model and data assumptions in AI/ML approaches. For example, if you look at Time Series Modelling (Estimation or Prediction) ...
Maeaex1's user avatar
  • 550
7 votes
3 answers
14k views

Understanding Locally Weighted Linear Regression

I'm having problem understanding how we choose the weight function. In Andrew Ng's notes, a method for calculating a local weight, a standard choice of weights is given by: What I don't understand is, ...
lte__'s user avatar
  • 1,320
7 votes
3 answers
4k views

xgboost: Is there a way to perform regression on rates/percentages data?

I have a dependent variable, $Y$, that is made up of rates/percentages data, so each value is between $0$ and $1$. I was attracted to the xgboost library because it allows focusing in on specific ...
Coolio2654's user avatar
7 votes
1 answer
657 views

Can one build linear models on "chunks" of the data set, if one can't build them on the entire data set?

Can one build linear models on "chunks" of the data set, if one can't build them on the entire data set? Particularly, I still have over 88k variables (features) left and one cannot do much with them ...
mavavilj's user avatar
  • 416
7 votes
2 answers
2k views

Possible harm in standardizing one-hot encoded features

While there may not be any added value in standardizing one-hot encoded features prior to applying linear models, is there is any harm in doing so (i.e., affecting model performance)? Standardizing ...
thereandhere1's user avatar
7 votes
2 answers
3k views

The Why Behind Sum of Squared Errors in a Linear Regression

I'm just starting to learn about linear regressions and was wondering why it is that we opt to minimize the sum of squared errors. I understand the squaring helps us balance positive and negative ...
stk1234's user avatar
  • 573
7 votes
2 answers
744 views

Why augmenting the training data with binary attributes works better for our dataset?

We have a dataset with multiple features ~400 where all of the features have a histogram as you can see in the following picture (sampled only a few) Our assumption We thought that this looked like ...
George Pligoropoulos's user avatar
7 votes
3 answers
682 views

Regression model with variable number of parameters in dataset?

I work in physics. We have lots of experimental runs, with each run yielding a result, y and some parameters that should predict the result, ...
JoseOrtiz3's user avatar
7 votes
2 answers
2k views

Theoretical bound - regression error

The Bayes error rate is a theoretical bound that determines the lowest possible error rate for a classification problem, given some data. I was wondering whether an equivalent concept exists for the ...
Pablo Suau's user avatar
  • 1,787
7 votes
0 answers
513 views

differences between LSQR and FTRL when working with very sparse data

I have a 2M instances dataset with millions of very very sparse dummy variables created using the hashing trick = ...
ihadanny's user avatar
  • 1,357
6 votes
3 answers
40k views

Which parameters are hyper parameters in a linear regression?

Can the number of features used in a linear regression be regarded as a hyperparameter? Perhaps the choice of features?
Vykta Wakandigara's user avatar
6 votes
3 answers
2k views

Correlation vs Multicollinearity

I have been taught to check correlation matrix before going for any algorithm. I have a few questions around the same: Pearson Correlation is for numerical variables only. What if we have to check ...
Payal Bhatia's user avatar
6 votes
2 answers
8k views

Is NN with no hidden layer is behave like a regression?

Is a NN with no hidden layer is behave like a regression? What we could say that NN without hidden layer can say us? ​ If we have for instance 20 input and 4 output and I have no true label, is it ...
user10296606's user avatar
  • 1,834
6 votes
1 answer
2k views

Why after adding categorical data the Linear Regression fails?

Based on a training set we applied a simple Linear Regression on some attributes that all were numeric. Now we have more attributes in terms of categories and of course we applied one-hot-encoding to ...
George Pligoropoulos's user avatar
6 votes
2 answers
4k views

Why does feature scaling improve the convergence speed for gradient descent?

From this article, it says: We can speed up gradient descent by scaling. This is because θ will descend quickly on small ranges and slowly on large ranges, and so will oscillate inefficiently down ...
Yandle's user avatar
  • 221
6 votes
4 answers
7k views

How to predict ETA using Regression?

I have a data from GPS in the form 1.('latitude', 'longitude','Timestamp'). 2.('latitude', 'longitude','Timestamp'). 3.('latitude', 'longitude','Timestamp'). I ...
user825828's user avatar
6 votes
2 answers
709 views

Confidence interval interpretation in linear regression when errors are not normally distributed

I've read that "If the error distribution is significantly non-normal, confidence intervals may be too wide or too narrow" (source). So, can anyone elaborate on this? When are the confidence intervals ...
Angadishop's user avatar
6 votes
2 answers
1k views

Gibbs sampling in R

I have the following model: $y_{it}=\alpha + x'_{it}\beta_{i} + \epsilon_{it}, \text{ } i=1,2,...,N, \text{ } t=1,2,...,T$ (1) $\beta_{i}= z'_{i}\gamma+\eta_{i}$ (2) with $\epsilon_{it} \sim N(0,\...
quant's user avatar
  • 353
5 votes
3 answers
19k views

Feature Selection in Linear Regression

I have a insurance dataset as given below. For which I need to build a model to calculate the charges. ...
deepguy's user avatar
  • 1,441
5 votes
3 answers
679 views

Problem with basic understanding of polynomial regression

I have an understanding of simple linear regression. Clear that results in a fitted line like this: However, studying polynomial regression is a bit of a challenge having some questions about the ...
Fredrik's user avatar
  • 987
5 votes
4 answers
270 views

Is the prediction algorithm absolutely the same for all linear classifiers?

Is the prediction algorithm absolutely the same for all linear classifiers and linear regression algorithms? As known, any linear classifier can be described as: ...
Alex's user avatar
  • 649

1
2 3 4 5
16